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ABSTRACT 
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conclusions reached through research and development activities. In 
the second section, the description o£ project activities that were 
conducted to develop the ISF guidelines provides a general overview 
followed by a detailed description of the major activities. The third 
section presents the findings of the investigations and analyses for 
each feature. The fourth section presents eight major conclusions and 
recommendations. A list of 14 references and a 26-item bibliography 
are also included, as well as a glossary of terms and a list of 
abbreviations and acronyms. Six appendices include a list of training 
sites visited, an interview guide used to collect data, a list of 
course documentation for various ATDs, an annotated listing of 
systems with advanced instructor support capabilities, a list of 
documentation on these systems, and a detailed comparison of internal 
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This report documents the research and atuilysls project that resulted in 
the development of Instructor Support Feature (ISF) Guidelines. The 
guidelines are intended to aid operational users from the Air Force major 
conmands. Simulator Systems Program Office procurement personnel, and 
contractors in the development and procurement of instructor support systems 
for future aircrew training devices (ATDs), During the 12-inonth technical 
effort, the Guidelines content and format were defined, data were collected 
and analyzed for inclusion within the Guidelines, and the Guidelines document 
was written. Thirteen advanced Instructional systems and ATPs provided data 
for identification and definition of ISF requirements. Volume 1 documents the 
research and development effort and presents methodology, results, conclu- 
sions, and recommendations • Volume II contains the ISF Guidelines. The ISF 
Guidelines is a "living" document. The current version of the Guidelines can 
be obtained from the Simulator Systems Program Office, ASD/WEE, 
Wright-Patterson AFB, OH, 
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PREFACE 



Sv«,.pI^pm^^T^. 1. '^^^^^ ^eP°'*= of the Performance Measurement 
System (PMS) Guidelines for Aircrew Training Devices (ATDs) project 
conducted under Contract Number F33615-84-C-0054, sponsored by the 
Mr Force Human Resources Laboratory (AFHRL). The project focused on 
aJd l^tTZjl ' M Support Feature (ISF) GuldeUnes to 

^ °^ requirements for ATD acquisitions. The 

Guidelines are published as Volume 11 of this report. 

A4 ?f°' y*^^ ^"'^ of AFHRL/OT provided technical 

direction during the course of the study. Mr. Craig McLean and Ss 
I t t ^ Simulator Systems Program Office made valuable 
contributions to the contents of the ISF Guidelines. vaxuaDie 

The authors wish to express their gratitude to the manv 
anra«r«L''''"°°^^\^' sites ^visited for their time 

Tu^tTonSs report: ^ 
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I . INTRODUCTION 



This report supplements the Instructor Support Feature (ISF) Guidelines. 
The purpose of the ISF Guidelines is to provide a conununications vehicle among 
those personnel involved in the acquisition process for a new training device. 
These personnel include operational users at the Air Force Major Conunands 
(MAJCOMs) who initially state training requirements, Simulator Systems Program 
Office (SimSPO) procurement personnel involved in the final specification 
definition, and contracting personnel involved in system development. The 
Guidelines specifically provide guidance in the development of specifications 
for the "instructional component" within the total simulation system. The 
research and development activities conducted in support of the development of 
the Guidelines revealed several interesting and notable results and findings 
with respect to the use of aircrew training devices. The purpose of this 
report is to document these conclusions and present the methods used to reach 
them . 

The following major conclusions were reached: 

1. A comprehensive front-end analysis of both student training tasks an^ 
instructor requirements is required in order to ensure that the instruc- 
tional system is designed to meet user needs. 

2. Instructor training in the use of instructor support features is 
needed. 



3. The concept of the "task module" has successfully been used to ensure 
that operational data are provided so that the resulting instructional system 
supports user needs . 

4. Instructional system data (e.g., maneuvers and procedures, displays, 
and performance measurement criteria) must be kept current with changes 
in training requirements and flight procedures. Provision for economic 
update and revision is cmjicial. 

5. The instructional system should provide for level(s) of automated 
control that support the specific training objectives. 

6. The Automated Performance Measurement feature should be designed as 
an aid, not as a replacement, to support the instructor in an evaluation 
of the student's performance of the training objective. 

7. The specification of instructional features should be based on 
functionality and performance from a user's perspective rather than on the 
latest technology. Current technological advances and current standards can 
then be incorporated to properly support these specified functional require- 
ments . 

1 
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f Guidelines should be continually updated so that lessons 
learned and proven technology from advanced instructional systems can be 
effectively transitioned into the operational training environment. 

These conclusions are addressed in greater detail in Section IV. 

BackgrouT^ d 

<;^«^J".^^^^'/^® ^^"1^° °/ ^^"^ Aeronautical Systems Division (ASD) 

stated a need for enhancing the instructor's capability to assess student 

canfbn?;?: ".^1,?'\.n^" '"P"^"*^ ^^"'^'"^ performance measurement 

Ro^rd liJ^ c identified by the Defense Science 

Board 1982 Summer Panel Study on Training and Training Technology. 

systems, incorporating state-of-the-art performance measurement 
technology have been successfully implemented in research and development 
environments and have provided valuable lessons. A means was sought fox 
capitalizing on this information for application in the operational training 
environment. Development of a set of guidelines addressing the design 
development and incorporation of Performance Measurement System (PMS) capa' 

renulS™/r^^ °i ""^^ S^^^elines was defined to include performance measurement 
?o^lc «nH f r P«ameters measurement, associated scores, and start-stL 
brfpfmr r instructor support requirements such as scenario control, 
briefing display of information, and debriefing. Associated computer hardware 
and software considerations were included as well. Instructor support 

to? « ent °"' ^"S ^^^r^,^^^-^ -'ly °n in the project as essential to insSu" 
tor acceptance and utilization of the PMS within an ATD. Thus it became 
apparent that the guidelines must feature all instructor support requirement 
with performance measurement as one of these requirements. The guWeU^e^ 
were therefore renamed "Instructor Support Feature (ISF) Guidelines . "The tern 
Instructor support feature" is used to describe those capabilities of the ATD 
that are specifically designed to aid instructional activity. The term 
"instructor support system" (ISS) is used in this document to refer to compute^^ 
based systems which support instructor support features. 

Prototype training systems have demonstrated that an ISS can provide the 
instructor with greater ability to control and monitor student actr^ity and 
^.ITJ^^l ^° "^^J ''^^ simulator a more effective training system. These systems 
have much to offer insofar as lessons learned during their development test 
and evaluation, and operation. The lessons learned from these systems a^ well 
as from other ATDs can contribute dramatically to the improvement of the 
specification of ATD requirements. F*"v«uient or tne 

. ^, single guidelines document could be used by the spectrum of 

individuals involved in specifying ATD requirements, it would servers a 
common basis for communication of need and would promote a greater degree of 
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mutual understanding. Current ATD acquisitions utilize specifications for 
instructor support requirements which are rudimentary at best. This has 
resulted in the design and implementation of aircrew training devices with 
features which are either too difficult for instructors to use or do not 
fulfill the training needs. These deficiencies result in low utilization 
rates by instructors, loss of productive training time, and ineffective 
training. 

In summary, the major goal of the effort was the development of a set of 
clear usable guidelines for instructor support feature requirements in future 
ATD acquisitions. These guidelines are intended to be used by MAJCOM require- 
ments personnel, SimSPO specification writers, ATD users, and contractors 
alike It is anticipated that use of these guidelines will result in well 
defined specifications that lead to the provision of useful instructional 
support capabilities within ATDs. Secondary goals include the provision of 
data which facilitate the further definition of mission tasks and instructional 
support functions and the promotion of more effective ISS designs. 

The remainder of the report is divided by section. Section II, Method, 
describes how the study activities were accomplished. Section III, Results, 
documents the findings of the investigations and analyses. Section IV, 
Conclusions and Recommendations, discusses overall conclusions derived as a 
result of these activities and offers recommendations for the future. 
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II. METHOD 



This section describes the project activities that were conducted to 
develop the ISF Guidelines. A general overview is followed by a detailed 
description of the major activities. The results of these efforts are described 
in Section III. The categories in Sections II (Methods) and III (Results) are 
presented in parallel order. 

General Overview 

The project goals were achieved using a three -phased study approach: 

1. Definition of design guidelines content and format. 

2. Review of simulator training requirements across a spectrxm of Air 
Force Commands, and review of the state-of-the-art capabilities of four 
systems which utilize advanced instructor support features. 

3. Development of the ISF Guidelines and a sample specification. 

Figure 1 presents the overall "roadmap" of the approach. The three 
phases of the study were, for the most part, time-phased, with some overlap in 
activities. 

The project began during Phase I with the identification of prospective 
content and format for the guidelines. In order to accomplish this, the 
specification process and relevant sections of specifications from recent ATD 
acquisitions were reviewed. In addition, a meeting with SimSPO and MAJCOM 
personnel was held to determine needs. A library of materials, including 
course docxunents, systems docvunentation, and ISF-related information, was also 
used to determine appropriate content and potential formats. 

Phase II encompassed a review of simulator training across a spectrum of 
Air Force Commands and a review of the state-of-the-art capabilities of four 
cystems which utilize advanced instructor support features. The review of 
these systems included both data collection during site visits and the review 
of course documents collected during Phase I. During each site visit, ATD 
training requirements, including aircrew training objectives, simulator 
characteristics, and instructor control and informational requirements were 
collected and assessed. 

Simulator training was observed at the following MAJCOMs. 

1. Tacttol Air Command (TAG): A- 10, F-15, F-16 aircraft 

2. Military Airlift Conmiand (MAC): C-130, C-141 aircraft 

3. Strategic Air Command (SAC): B-52, KC-135 aircraft 

4. Air Training Command (ATC) : T-37, T-38 aircraft 

Locations and dates of th^^^se visits are identified in Appendix A. The interview 
guide utilized during these visits is included in Appendix B* 
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Training documentation including course syllabi, task and objectives 
documents, instructor guides, and simulator manuals was collected. Complete 
documentation was difficult to obtain. Over 30 documents were reviewed 
for information regarding aircrew training objectives, simulator character . 
istics, and instructor control and informational requirements. A listing of 
course documentation acquired and reviewed is included as Appendix C. 

Four additional training systems have been built as modifications to 
existing ATDs and were included in this review. These systems were designed 
to specifically provide advanced instructor support capabilities and include: 

1. Automated Flight Training System (AFTS^^ for F-4E and A-7D aircrews. 

2. C-5A Performance Measurement System (C-5A PMS). 

3. F-14 Instructor Support System (F-14 ISS). ,.oott 

4. Air Refueling Part Task Trainer Instructor Support System QAKl-il 
ISS) for B-52 aircrews. 

A brief description of each of these systems is provided in Appendix D. 

Descriptive data on these systems were collected and reviewed. These 
included functional specifications, design documents, operation manuals, and 
program source listings. Test and evaluation data were also reviewed. Appendix 
E provides a listing of the documentation reviewed. 

Interviews were conducted with personnel who were directly involved in 
the development of the systems. The purpose was to verify the information 
contained in the documentation and to derive lessons learned regarding system 
functions and hardware and software implementation. Visits were made to 
selected MAJCOM sites to observe training and to determine user attitudes 
regarding the design and implementation of the advanced instructor support 
features. These visits are included in the Appendix A listing. 

The third phase culminated in the writing and production of the actual 
guidelines, including a sample specification and procedure by which the 
specification was generated. 

Data management was a major concern throughout this project. Preliminary 
information and data obtained as a result of the review of training and 
systems documentation were entered into computer-based files. These tiles 
contained data describing each ATD, the aircraft, instructor support features, 
tasks trained, performance measurement, and scoring and information sources. 
After site visit information was obtained, these data were reworked into data 
files which describe each ATD in terms of training objectives, instructor/ 
operator station (lOS) type, lOS control, lOS displays, performance evaluation, 
ISFs, and comments regarding lessons learned. 

Project notebooks for internal use were utilized to organize correspondence 
and data relating to the project. A project library that was maintained 
included over 100 references, including recent research publications that 
address the issues of ISF design and use. These sources are identified in 
Appendices A and D and in the References and Bibliography sections. A glossary 
of ISF- related terms and acronyms were compiled and updated periodically 
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Because five of the nine ATDs investigated conduct pilot training only, the 
scope of the analysis included a comparison of pilot flight station tasks 
only A detailed review of training documentation, including the simulator 
trailing syllabi and instructor guides, was made in order to identify tasks 
trained on each ATD. This documentation describes the general training 
scenario and the specific training objectives for each event In >nany cases 
however, a description of the training events on a task- by -task basis is not 
provided. Extensive interviews with instructors were conducted to obtain 
further information about which specific tasks were monitored during simulator 
training sessions. 

A. listing of training tasks by phase of flight was compiled for each 
ATD The tasks were then grouped by type into the following categories. 
Normal Flight, Normal Procedures, Emergency Flight. Emergency Procedures. 
Tactical Flight, and Tactical Procedures. The tasks which are monitored in the 
four systems were then identified from existing documentation and grouped 
into the previously defined task listing. To facilitate comparison and 
analysis of these training tasks, a task-listing matrix was generated; this 
matrix has been included in Appendix E of the ISF Guidelines. 

riomparison of Internal ISS O aracteristics 

The four systems representative of the current capabilities were compared 
on their ability to monitor, to compute performance measures and score and 
to trigger other instructional support actions. Common and effective functional 
characteristics were identified to provide guidance for future ISS design. 

TSS Hardware/Software Im plications 

The ISF requirements of these four systems were then analyzed from a 
systems engineering, hardware, and software perspective. This analysis 
provides reference for future implementations. 

Definition of Guid eltpes Format 

Several alternative guideline formats were considered for presentation of 
the guidelines information. A literature search of format types used in 
other design guides/handbooks was conducted so that an effective framework for 
presentation of the content of the guidelines could be designed. Three basic 
format types were considered and are briefly described as follows: 

1 Checklist . This format type is used in the AFSC Design Handbook 
(1977)* Checklists are provided to use for establishing design requirements 
and for verifying that the requirements have been met. The intent is to 
ensure that all applicable design factors have been examined and that all 
problems were resolved or otherwise determined unimportant to the design. 
Each checklist item in the handbook cross-references to another section 
entitled "Design Notes" which provides coverage of a specific topic. 

2 Narrative . The particular format type under consideration was one 
used in Caro, Pohlinaim, and Isley (1979). This format was intended to communi- 
cate information on instructional features to engineering and other simulator 
design personnel. It consisted of the following elements: 
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Feature 
Definition 

Purpose and Intended Use 
Function Description 
Concurrent Events Description 
Feature Diagram 



^' HQ^^I Specification with Accomp Anvinp HAndhnnV this format ctvlo 

annlinLinf p ""u ^^^^^ ^"^^^ specification to a specific 

application. For each paragraph and subparagraph of the specification the 
following sections are addressed in the handbook- specirication, the 



Rationale and Guidance 
Performance Parameters 
Background and Sources 
Lessons Learned 



s^«nH^rH \! h"^/""^ format layouts were considered for the Guidelines: the 
^n^nHI header/paragraph layout that is used in most documents and Se 
Information Mapping® writing style described in Horn (1983) tSat' oSers a 
more unique visual presentation. The Inforniation Mapping® style provides 1 

m'm?"^^^?""''^ "''"S ^"^"^"^ ^° the'Ltfrial '^ Thrse labels 

highlight the structure of the information, making it easy for the reader to 
locate relevant detail. Because information is presented in mod^llr^^ts 
changes and updates can be easily accommodated. moauxar units. 

Guidelines samples were prepared in these two alternative layouts and 
then presented to lOS Wdrking Group members v?ho. as representatitrof Tuidelinef 
users, selected the Information Mapping® style as the preferred Uyout 

The feasibility of on-line computer format alternatives for ease of update 
was also considered. Final delivery of the Guidelines Included tbm ^^rnL^^i 

Guidelines Develop ment: 

The development of Guidelines content was a iterative process. Durine the 

the n\e1s of ^'hl^G^d^^'^"''"' "^^-^ several times to Jet 

i '^^^^ Guidelines users. The document is organized into four 

times ?n't-r'^Tn"''^°" ^""^"^^''^ "° "^'^ by different users at different 
times in the ATD procurement process: uj-j-j-etenc 

I . Overview 

II. Instructor Support Features 

III. Selecting Instructor Support Features 

IV. Providing Operational Information 
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nevelooment n f t <;f neftnltlons (Section II, Instructor Support Features) 

Based on an extensive survey of research publications which address the 
subject of ISFs, specifically Caro, Pohlmann, & Isley, 1979; Semple, Cotton, & 
Sullivan, 1981; Polzella, 1983; Leaf, Fitzpatrick, & Gunzburger 1983; and the 
nresent analyses, sixteen instructor support features were identified for 
inclusion in Section II of the Guidelines. Originally, only "advanced instruc- 
tional features" (e.g., performance measurement, scenario control) were 
intended to be addressed. Advanced instructional features are those features 
which increase the instructor's efficiency and effectiveness by reducing the 
•,7orkload and providing support in the total instructional process of simulator 
training. However, it was felt that more basic ISFs (e.g., record/replay, 
freeze) should also be included. It should be noted that the term "instructor 
support feature (ISF)" was specifically selected for use because it so closely 
describes the purpose of these features. An attempt was made to provide a 
concise definition for eath feature, describe its instructional value, a.lst 
additional considerations to be examined when fine-tuning the specification, 
note related features, and provide examples and lessons learned based on past 
experience. The content of the ISF definitions is based on the analyses 
results and a review of the research material cited. 

Drafts of the definitions were discussed at a working meeting held 
in April 1985 at Luke AFB. The meeting was attended by personnel from the 
4444th Operational Squadron, as well as from AFHRL and SimSPO. Feedback was 
sought to determine whether they met the typical user's needs. Initial 
response by this representative group of operational users was positive and 
suggestions for improvement were incorporated into the Guidelines. 

The "Task Module" Concept 

A control mechanism successfully implemented in the four systems with 
advanced instructor support capabilities was that of mapping training tasks 
into modular data files referred to as "task modules." In these systems, task 
modules have successfully served as the means by which ISFs are implemented. 
Because of this success, the task module concept has been introduced in the 
Guidelines (Section IV, Providing Operational Information) as one approach to 
implementing a data-driven instructor support system. 

Task modules are presented as tools to be used by operational users 
which provide a framework for specifying ISS requirements so that required 
training support functions will be provided by the ATD. For example , task 
modules identify the conditions triggering or terminating a training objective, 
and define the appropriate aircrew performance measurement procedures and 
information displays with respect to that objective. Refinement of information 
contained in task modules continues as training requirements are defined more 
explicitly. Ultimately, this information is translated into data files and 
modular software programs by contractors. Thus task modules are the bases of 
an approach to modular software architecture from which an ISS could be built, 
which would control the operation of the system. 
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IS? Guldel -fri es App enrfioefi 

by engaging 'I'n thL Voces/^fV^oe"" f oTZw^lL^/'i' " 
re,ulre,.e„t, for future ATD procurem^ntf^L irlvet''^ °* Instructor support 

generated" »ouwtt !d:«med*" rl« ISase i*° T"'' =P-i«catlon was to be 
by the Air Force Mas the c!?30 5h. V, ^° ^^l"" designated 
Dual Role Flehter TdrM p. i , "le"":!?" Mas then changed to the P-ISE 
until the end 'Jt™". o^t"' upgrade was not made 

however, have a positive l^n^.t t *=^\fdentification of the system did, 
serve a useful pu:^ose W ^naS;^;^ specification to 

affect an actuarpXlrement rather '°'^^^"S/°''"»ent that would potentially 

that vouid si.piy^::^s?r;?; t^^^p^jj^^.Tri? srcuid:iJsr'^"^^^ 

Squadron'sTl'eT Ji'^ "^"^"^ Instructional Systems Development (ISD) 

irotrr^o'-lLn'tffy^^^^^^^^^ were heldTt'luki^pi 

ments. Based on their i%':t\, 'fuSL^T^^^^ TT." ^y^'^" 
compiled, similar in format to the descriSf ions f^nnd f J^"^ objectives was 
matrices. Identification of ««oW>. f ? ^ ^" ^'^^ commonality 

of requirenients in tex^s of briefer. f ^"^^^^'^ definitioj 

monitoring, and debrTe fine ron^^;^, '^^^f^^"' instruction, 
hypotheticli training events 'nd ^l^^ r^^ ^"^^ sequentially mt^ 

training exercise enabUd i 't^cto^^ '^^^ "^^^^^'^^ °f ^ 

further ' ^^^^^^"^ instructor support requirements to be defined even 

weU;^^T;;:s: Zlt'^:.^'^^^^^^^^^ ^o-t structure as 

et al (1983^ rh^ Un.in ^ " V Qiy&a) and the draft specification by Uaf 

unchanged f'r;«'^revi:i: ^prifUatTo'n"" '''' specification rLltns 

differed. Using" tTe T«c?dure"\lrTbed S~in "ofThe ^ iHJ"'^ 
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Appendix f^. Task Commonality Analvslo . The matrices contained in this 
appendix provide a listing of general tasks currently trained at nine ATD 
sites. Although this table is provided to show commonality of tasks across 
several different training sites, It may provide guidance in the development 
of a list of task modules for future ATDs. 

A ppendix F. ISS Implementation Conside rations. ISS cost and Implementation 
guidelines have been presented as appendix material for SimSPO personnel who 
have technical backgrotinds but limited experience with ISS implementation. 

Appendix Samplfi Task Modules . Representative samples of specific task 
modules were developed to provide specific cases for reference by those 
Involved in the specification process. 
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III. RESULTS 



This section presents the findings of the investigations and analyses 

efforts. These results are described under the following headings and are 

presented in the same order as described in Methods (Section II) immediately 
above : 

1. Current ATD Acquisition/Specification Process 

2. Instructor Support Feature Analysis 

3. Task Commonality Analysis 

4. Internal ISS Analysis 

5. Hardware/Software Implications 

6. Guidelines Format Selection 

7. The Final Product The ISF Guidelines 

Cui ^rent: ATD Acquisition/Specificati on Process 

Acquisition of ATDs is handled by the Deputy for Simulators, SimSPO, of 
the Aeronautical Systems Division, Air Force Systems Command (AFSC/ASD/YW) . 
SimSPO is involved in specifications for acquisitions that range from training 
programs to products and equipment. Some of these acquisitions have included 
ISSs for ATDs. The SimSPO follows established procedures from ATD project 
inception, through contract award, and to final transfer of the ATD to the 
using Command and the Air Force Logistics Command (AFLC) . These procedures 
are mandated by Air Force Regulation (AFR) 800-2 (1982), Acquisition Program 
Management . 

The SimSPO, however, is not always involved with acquisitions. For 
example, simulator "refurbishments, " which may include changes to the simulator, 
can be procured through the AFLC. AFLC, with Ogden Air Logistics Center 
acting as the implementing agency, is responsible for simulator modifications 
and maintenance after the initial acquisition. The regulation governing 
Modification Program Approval and Management is AFR 57-4 (1983). 

Need Identification 

Training requirements are initially identified by the MAJCOMs using the 
ISD process, an approach to the analysis of training requirements and develop- 
ment of training systems. This includes performance of a task analysis of the 
missions to be trained and media selection. Relevant regulations include 
AFR 50-8 which requires that the ISD process be utilized in the identification 
of training requirements, and AFR 50-11 (1977) which requires that all training 
equipment be designed according to ISD methodology. 

Acquisition of a new ATD is formally initiated by a Statement of 
Operational Need (SON) , a statement of training requirements generated by one 
of the MAJCOMs. It is a formal docxjunent which identifies an operational 
deficiency and states the need for a new or improved capability. In a SON for 
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stateVvi"; gfnerauT "iTuv '"""'V,*' "<l-"re,„ents „„y be 

win be rVir"r""so"/^^\\^lfr"^o"'?he"AID ''o^'Vhe^'Jr"^ 'll" 
provides more substantial rto^^>^^ u ' ^^"d, usually 

Therefore, the lever o? deta^^^^^ operator control and ISF requirements^ 

of the acquisitior of a ma1or sv^^^^^^^^ substantially among sONs . In the case 
the Air Force is reoufrpH I ^ ' ^""^ approval by the Secretary of 

(MENS is'Jece sarr T^^Air%^"'e'R^^^^^^^^^ ' "'r.'^" '^^"^^^ ^"^^ ^^a^eL^t 
Of the SON and MEN? is m%S^n":aLTiro1"oSr^S^^^^^^^^ ^^0^?"^"^"°" 
Concept neve Xopmept 

other^i;?:^- p"--P-~^ - 

Se^eXic^r3e. ^oyistfcrc^rts^rn^^T ^'^^^ fnf'exSr^fL^e^^ 
Command notes their input to refine draining, etc. The using 

SimSPO and the using Sand begin to wo^^^^^^ ^^^^ ^i-' 

specifically. After suVgested ch^"-^ together to define user needs more 
included, the final version of the In,. incorporated and cost estimates 

validation. version of the SON is sent to HQ USAF for evaluation and 

estima'ted'risourrerro ^atVfr'th:'^'^""' identifies the 
Commands so a recomended coirse of ac^^^^ and requests review by the other 
mination of priori^ra^Uing for the SON ^r'^ENs' T'ff " ^ 
Management Directive (PMD) bv HO U?If nL?Ji ni issuance of the Program 

SON or MENS has been Vali^ted or annrov// ^ =°""^n«<i ^s to whether the 
approved. It is at this n«?^^ approved, in whole or in part, or dis- 

Jhil -r, . point, upon issuance of the PMD and AF<;r Vr.^ tl 

that the program is assigned to SimSPO Thp <Hm<tvn f 

(PM) whose role is to .rnirfp t SimSPO assigns a Program Manager 

the i^!eZtl'4''^d%"«L%^rt?„T'""' f"'"" direction to 

re^ouroes to =«i"/thfoper/tiS!l°Zr commitment of 

works with subject natter e™e"r?i„m t?' • "'""^^ *^ fMD, the SimSPO 

operatlonel id ^HA>.7Z''Zot' Tt^Z^Zrlx'^^^r^'T^ '"^ 
preaches. The SlnSPO relies heavllv or, ™ V< aitematlve design ap- 
Test and Evaluation (OT4E) and other n^rt.^ ="^'"""8. Op«»tional 

actively m the acqulsiSTn li?f cyae 'aU farti " P""clpate 

system design, operational, anf support coice^rdeXmLt"'''""' " 

neerltr^%^t=%ro%^V':iJfri:^^^^^^ ^,1^/1° ' ^^^^^ 

functional .e,uli..ents. tTT Il\^"ro?cr"iL?%:r«^"V4rato^7 
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training psychologists and engineering personnel specializing in simulation 
research may make contributions during all phases of the acquisition process. 

Generation of Specif ication 

A team composed of personnel from SimSPO and also operational training, 
management and engineering personnel from the using Command is tasked with 
developing the actual ATD Prime Item Development Specification (PIDS). SimSPO 
personnel meet with user representatives via site visits to determine how the 
system will be used, facilities requirements, etc. A draft specification is 
prepared in accordance with MIL-STD-490. This draft is reviewed in detail by 
the users as well as by the Director of Requirements (DR) and Director of 
Operations (DO) at the MAJCOM. In some cases, a draft of the specification is 
sent to industry for comment. After modifications have been made, the final 
version of the specification is published. 

Problem A feag Identified 

The investigation of the ATD acquisition/specification process and;; 
discussion with MAJCOM and SIMSPO personnel pointed to several problem areas 
relating to ATD acquisition. Identification of these problems enabled the 
definition of a guidelines content that would meet user requirements. 

Because equipment and -training requirements are often developed con- 
currently, front-end analysis does not always precede procurement. Typically, 
the simulator is procured first, and then the training is defined. ISD goes on 
during the acquisition process but, unfortunately, does not always drive the 
requirements. This has led to the design and implementation of systems that 
often do not support the intended training objectives or meet instructor needs. 
If, instead, the operational users would clearly identify their requirements 
and communicate them to the specification writers at the outset, specifica- 
tions could be generated that would more accurately reflect their needs. 

In an effort to reverse the process of procuring simulators first and then 
defining training, the SimSPO is attempting to move towards contractor -provided 
training programs. In the future, a contractor may be responsible for the 
entire process, including the front-end analysis, development of necessary 
media, and the operation and maintenance of the resulting program. The SimSPO' s 
role will be to oversee the process. This trend has significant implications 
to the development of ISSs in ATDs. Specific guidelines for specifying 
requirements based on past procurements would provide a repository for lessons 
learned to benefit all future participants and would promote sharing of 
information. 

Another problem area is the lack of personnel who are properly trained in 
identifying ATD training and ISF requirements. This is partly due to personnel 
turnover, unfamiliarity with state-of-the-art technology, and the lack of 
adequate guidelines for selecting ISFs of new procurements. Unfortunately, 
what happens « too often is that an ATD procurement is based on the dictates of 
an individual, who is only familiar with one particular device, rather than a 
result of an effort based on "corporate" knowledge. In other cases, there is a 
tendency to over-acquire "just in case." The resulting ATDs, then, are either 
insufficiently equipped with features or are too complex to utilize fully. 
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understanding as to each feature's purpose and intended ^se' ' 

dir«c!?on i**"' P'^*^^^"^ specifications have not provided enough 

airection to the contractor in regards to user ^r•o^T,^r,„ »j * Fi.wviuoa enougn 
feature rAo..4T.fl™««*.„ t teisai-ua co user training and instructor support 

Is S^^W^ if '» »f"y/«"«. the description of required capabiXi?Jes 

i:ve"nirco.p?etel'^^^^ implementation of^ATDs ti:t 

simply list desired A?S factions without re°/ardTo' that 
exercise those functions an^ without rega^^^^ 
may produce a "feature- rich" ATD with unusable features 

ISF gwtd<^llnes TntAndBd Atirfj«^ ce And ^ J^A 

«S?e„%r^curers *o «e^ask' d ?lTh V„.c?? T'^"' J*" specification 
the coni?aotors tSLill A?Ds '■Pedlylng requirements In the PIDS, and 

selecting t<5p« t./> * Fi-^^v-L^c «ssisi:ance to operational personnel in 

tS^/li:!d"p?„vJL^rV:^:t onTl^Ss-Jr^/trn^r-l'srs t sri'fl 
^e'rftlona'r"^' 4'e%tld:rin "°„^uW p 

ri7iStKrhr:ied"h^u%^;nrdetiot:i-^- — " 

Ingtructor Support FAa ture Analysis Resultc 
The purpose of the ISF analysis was to compare ISF utlll2„^^nr, ^ 

:s:ncinrF: tsuit :??hrditr%rr the"Lur^^^rteirw??h' 

this section Tables ^ and 2 confr^ described in 

Section II of the iSF Guidti?.! ^ supplementary information. Refer to 
additional lessons UaLj '^"''^'^ definitions of each ISF and 
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Tabio 2. Cbaracteristjcj/Fflatures of Four Systfimi 
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Parameters and maintenance of 
parameters values within windows 
Crew* member issessment 
Crew coordination assessment 
Mission assessment 



N/A NAVY 
F.14 

Maritime Air Superiority Fighter 

ISS Strap-on to F I4 0FT 

Normal/Emergency Procedures 
Normal/Emergency Flight 

1.5 hours 

IP offboard station 
Touchpad on display 
2 19" graphics displays 
Remote Terminal 
Touchpad on display 
2 19" graphic diiplays 



YES 

Fully Automated 

Semi Automated/By Objective 

Can make runtime changes 



Wind direction and velocity 
Air turbulence 
Carrier speed 
Sea state 



Automated 
Manual 

Operating areas, approach/dep 
plates 



SAC 
B-52 

Strategic Bomber 
Part-task Trainer 
Inflight refueling 

1.5 hours 

7 onboard stations w/PEP panel 

9" b/w monitor 

1 offboard station 

PEP panel 

19" color monitor 

Remote offboard station 

Touchpanel 

19" color monitor 

YES 

Fully Automated 

Semi Au torn ale d/Gy Objective 

Can make runtime changes 

A/C model and positions 
Tanker offload 

Tanker/receiver (T/R) flight parameters 
Visual eyepoint 
Grots weight, etc. 

Manual only 



T/R position: overhead, vertical, 
envelope views 



GCAs and Carrier Controlled Approaches Runtime graphics 
A/C historic and present position Altitude, Azimuth, Speed, Throttle, 

Total Fuel Flow Profiles 



Scores task modules in realtime, 
Can make realtime change. Total 
grade calculated; instructor can 
override grade 



Scores task modules proficient/nonproficient 
Evaluation Template (minimum 
proficiency and acluall 
Instructor can override grade 



32 



ERIC 



Table 2, Concluded 



AFTS 



C-5A PMS 



F-14 ISS 



OCEOURES 
>NIT0RINC 

TA STORAGE & 
lALYSIS 



lEFlNC UTILITIES 

rORlAL 

:EZE 
3RIEF 

lirdcopy/Prinlojt 



«OTE GRAPHICS 
UY 

JLATOR 
ORO/REPLAY 



B 52 ARPn ISS 



NONE 



Viluei of each performance tneasura 
Mved; Hitiorkal recofds of (raining 
and pflrformance: Interrogate feature 

Peplav uied for pr^miuicn brief 

NONE 

YES 

YES 
Offline 
CRT diipiay 



YES 

Data Retrieval 
Data Analytit 



NONE 

27 pages of help dltpUys 

YES 

YES 

Orrilne/runtime 
CRT display 



Performance Summary at erKl of exercise Crew Performance Ats«»n>ent 



YES 
NONE 



NONE 
NONE 



YES 



Samptei. proceites and 
records aircraft parameters; 
Performance Summary 
Training Seition Description 

Can call up aircraft lyitem 
description and procedures 

Tutorial to review ISS functions 
•nd features 

YES 



YES 

Offline/runtime 
CRT display 
Performance Sommery 
Exercite Summary 

YES 

NONE 



YES 

Oms Summary 
Comperallve Misiion 
Data Analysis 

Triining Summary iviilable 

Help information: ISS lyitem opera, 
tion, syllabus. Uaining summanes 

YES . 
NONE 



YES 
NONE 
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Summary of AriflTy. gls bv Ffta^ tirn 

The following is a stuninary by feature. 

gggnarlP gonm i. The systems which had fully automated scenario control 
included the 6-52 Weapon Systems Trainer (WST) , ^he F-16 Operational Flieht 

IZ'ZJ'^^' ^""^ ^y"^^- °- of tre differencL'Ji 

the operatic, of these systems is how the missions are generated. For the 
AIDS. ger.v.^^Lon of mission scenarios requires users to have data entry and 
programming skills. For the four systems that have advanced ISFs the trSn^^g 
objectives have been preprogrammed into a database and allow for easy selection 

in^tra!L"\rTrocesr.""'°^ '""'^"^'^'^ ""'^^"^ ^'^^^^"6 - 

V^yj^bleg qon t r o l . it was observed that the manual 
selection is besC suited for informal training (e.g., continuation training 
instructorless practice, and review). Simulation vfriables were available oA 
all of the systems Visited. The amount of usage depended on the accessibility 
of the variable and whether a device technician was available during training 

the ^^1^11^^''°^ variables by re-initializing the simulator seemed to break 

moL fn. ^ ^ "i"* detracted from the realism of flight. It was used 

are practiced ^^^S^^^on training where approaches to different airfields 

mode ""i?J«'L!^L^''?/°^''*°".!? B.52 WST when operating in the "mission 

mode. This system also provides for manual control of certain variables and 
allows flexible instructor interaction. ti-aDi.es ana 

MalfvngttPn Cgn^rPl . The automatic feature is rarely used The main 
reason given was that it did not allow the instructor to tailor training to 

dritTion'X" n malfunction activation a^d 

deletion did not correspond to the desired scenario. In particular the 

^rafned.^"^"^''^ malfunctions do not correspond to the syllabus being 

Th« B.52 ARPTT and the F-14 ISS provide a feattare whereby the instructor 
may select a set of malfunctions during the initialization process i^ese 
malfunctions are then grouped together on a special menu and may be readily 
accessed during the exercise for actuation and deletion. This feature is 
being used on the B-52 ARPTT ISS. toai.ure is 

ESEfisljian. Repositioning the simulator to a specific location was used 
on all devices visited and was mostly used for repetitive training (e e 
approaches) . This feature is accomplished in many different ways and in var^ine 
degrees. The most versatile method was seen on the A-10 ATD where the simulator 
can be positioned anywhere utthin the active geographic graphics display by 

tt .i^'^^^^ ^ ^^f^'l'^.^v^^? " P""' Repositioning in the A-10 simulatoj 

may also be accomplished by bearing and distance from a fix. latitude/longitude, 
or by identifying a previous position by a "snapshot Initial Condition (I C )•' 
The most common way to reposition was accomplished via an I.C. reset' ' The 
A-10 may be over-designed for the training requirement, however. The I C ' reset 
may be somewhat restrictive, time consuming, and difficult to access 
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The B-52 ARPTT and F-14 ISS have a feature whereby the trainer may be 
repositioned to the beginning of any flight -training objective (e.g. pre-contact 
position, initial approach fix). This feature is currently used in the ARPTT. 

The repositioning feature on the F-14 OFT was somewhat user unfriendly in 
that repositioning to the end of a runway would cause a crash condition if the 
landing gear were not down. 

^QS Display Control and Formatting . Selection of displays was observed 
to be both by Instructor selection or by automated actuation. In most cases, 
where the aircraft was geographically referenced on a display, an automated 
feature would provide the correct reference. For example, in all cases when the 
simulator was repositioned to the beginning of an approach, an approach 
display would automatically come up. In all cases when a geographic plot was 
being displayed and when the aircraft flight path approached the edge, the 
display would change to the next appropriate display. 

With respect to displays other than geographic referencing, the instructor 
has to manually select any display associated with procedures and or cockpit 
activity. In some cases, the cockpit controls were separated by relative 
position in the cockpit. In other cases, display of controls was by aircraft 
system. This separation does not necessarily provide total feedback with 
respect to certain procedures so instructors had to use some other "work 
around" method to evaluate performance. An exception to this is the ISS 
systems that automatically select displays appropriate to the current training 
objective. 

Those systems that provided checklists and procedures on special displays 
were not often used because the checklists and procedures were outdated. 

Total Sygtem Fyeege . All systems observed have this feature, and it was 
tiaed in varying degrees depending on the type of training. At the Undergraduate 
Pilot Training (UPT) level^ freeze was ttsed extensively by the instructor while 
providing direct feedback and corrective action. It is rarely used in total 
mission training. 

The Crash Override feature was found on all devices and, in all cases 
observed, was always left on. 

Partial Freeze . Partial/parameter freeze was found on many of the 
devices but was only used at the UPT level. Instructors expressed that there 
is little training value for this feature at the MAC/SAC/TAC sites. 

^ut j iomate^ Slmul^ator Demonstration . This feature was found on many of the 
devices but was only used at the UPT level, and then not very often. Instruc- 
tors expressed that this feature may have some value, but that they would 
rather use the simulator time for "hands-on" training. 

gli^tulatoy Record/Replay . This feature was found on many of the devices but 
was mostly used at the UPT level and then only by certain instructors. Other 
instructors expressed that this feature may have some value, but again, they 
would rather use the simulator time for "hands-on" training. This was espec- 
ially true at the MAC/SAC/TAC sites . 
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HQrdcopv/pyj,nt?9.vit;,. This feature was found on i • ^U^r.,,.., ^he 

systems required that the system be taken down n il-.. runtime programs 

prior to providing the hardcopy output. This was obsorvcd to be restrictive 

^?>.^°i«!, K^ri^i"^; ^"'^ ^^'"^ °°Py ""^de. the instructor has 

already debriefed the student. Some of the devices rRqulrcl ihnt- che display 
being hardcopied be frozen while the output was produced, fhia may be disrup. 
tlve with respect to real-time feedback. In most cases, this feature was not 
used often by instructors. The reasons varied from not knowing that it 
existed to not knowing what to use it for. 

firo<?eduyeg MonltorfnE . Some of the systems, including all those that 
utilize advanced ISFs, have this feature. However, it is not used much 
because the procedures are quickly outdated due to the many changes (e g 
aircraft configuration, local course rules, and ATC procedures). 

, Augoffgt^i;! Peyfpinnange Me^syrem^nt . Some of the systems have a parameters 

?r^?';„"^Hfjlf''''f; ""^y given were that 

it is to difficult to set up and that the results have little relevance to 
the objective being evaluated. 

Some of the WSTs have a feature called automated performance measurement 
where bomb drops and missile shots are scored. This feature is not used in 
.u""" ^'^^ because instructors feel that the basic simulation does not 
provide the cues necessary to properly launch the weapon. The F-15 missile 

intercept procedures and shots made beyond visual 

range (BvR) . 

.o^^r advanced instructional systems have a more comprehensive automated 

performance measurement feature that evaluates performance by training oblec 
tlve. The B-52 ARPTI ISS has just recently instaUed this feature - hoSeier . 
S?S® .llr?^^ ^ feedback with respect to operational usage. The 

AFTS^ scoring has either been used or not used depending on Command support 

weatSS is bad":nd°"i :;'" ""^^ frequently In Alaska where 'the 

weather is bad and pilot proficiency in instrument flying is critical to 
IttV ff/liS^^^: <5CAs are not used very much at Davis-Monthan where there 
may be 1 day in the year when an instrument approach is required and probably 
never to the field minimums). ptooaoxy 

5^ ^"'^ ^'^t ''^''^ designed primarily for R&D purposes, and no 

operational data were evaluated in this effort. f f . 

. Pata Storage and Analysis . The C-SA PMS has the capability to store and 
analyze scoring data. Operational evaluation data are not yet avalLbJe 
lllT^l this feature was only recently installed on the B-52 ARPTT ISS. no data 
have been collected with respect to operational usage. 

^ ^^r'^^J"^^^^^ Replay . Systems which provide this feature are the F-14 
and B-52 ARPTT ISSs. The F-IA ISS was an R&D development system and the opera- 
1 7" minimal. The ARPTT Briefing/Debriefing console has just 

recently been installed. However, the operational feedback thus far has been 
very enthusiastic with respect to the remote graphics replay capability 
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The AFTS®also provided for viewing of replay of exercise geometry and con- 
troller voice messagea for pre-mission briefing or post- training critique 
purposes in an adjacent briefing area. 

Tutorial . The only system which has this feature is the F-14 ISS. Not 
enough data were collected from either the user or from a research perspective 
because this feature was Installed just prior to re-hostlng the main computer 
of the simulator. Unfortunately, this re- configuration made the ISS inoper- 
ative . 

The B-52 ARPTT has a HELP function which provides the user with system 
usage Information which may be accessed upon request during briefing or 
debriefing. The C-5A PMS supports HELP pages. Not enough information has 
been gathered to make any comments on this feature. 



Conclusions 

A Scenario Control feature would be of value to all of the devices 
visited in that It would reduce the instructor workload during instruction. A 
fully automated Scenario Control feature (programmed mission scenarios) would 
be of greater value to long sessions, as in MAC/SAC, and to a lesser extent in 
ATC where a console operator, because of the design of the system, would 
provide the support. Programmed mission scenarios are most appropriate' for 
evaluating progress (e.g., checkrides) where standardization is important or 
for total mission training practice, but during normal training sessions, they 
do not allow for instructor flexibility and therefore limit training effective- 
ness. 

During many of the training events, the instructor should have the 
capability to tailor training to the needs of the individual student. Semi- 
automated scenario control allows the instructor to create a tailored mission 
to meet student needs and provides support to aid the instructor during 
training. Flexibility (e.g., instructor interaction with the system) during 
training is also essential. The instructor should be able to modify variables, 
insert and remove malfunctions, and move forward or backwards in the profile 
to satisfy basic instruction and student progress. 

The instructor wants flexibility and therefore resists automated mal- 
functions that cannot be changed and automated performance measurement that is 
rudimentary or inaccurate. Unfortunately, the operational deficiencies of 
some of these features have alienated instructors. Improvements in ISF design 
and instructor education as to their purpose and intended use should lead 
toward instructor acceptance. 

Software in the simulator should be up-to-date with respect to the 
aircraft. Data relating to aircraft procedures, for example, must be easily 
modifiable by an on-site maintenance activity if the procedures monitoring 
feature is going to be utilized and appreciated by instructors. 

Most of the ATDs reviewed provide automated performance measurement of 
simulator variables and display raw performance data. The ISSs provide 
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automated performance measurement by training objectives. The focus on 
ZtTlt^T/^ objectives rather than parametfrs ias observed to provide 
Tnitrrt ^ override feature would help diJJnish 

instructor resistance to the idea of automated performance measurement 

t-^^o of other demands on instructor time, months can pass without anv 

swiir A T^^?? difficult for the instructor to maintain Ss 

onirite ^>, ""^-f'i^ndly interface design that allows the instructor to 
reSesher trSnl^^ with minimal training or a tutorial built in the system for 
rerresher training would help solve this problem. 

Uh^^o^5^''"''^^°" on utilization of ISFs is usually informal and on the lob 
While discussing ISFs with instructors, it was noted that many of them were 

:pe":te tLr'afd^i'^^H"^ ^"'^'^'V ' ^^^^ Lt ZT^To 

«nn1?!^ I J " '^^^ "o*: ^ow the feature could be 

applied to training. Some instructors viewed the simulator as a substitute 
for the aircraft on the flight line only and do not take advantage oft^e ATO 
be «tf training system. The maximum potential of ATDs wiS only 
be attained when instructors are provided with the proper trJ^L^iZ L 
usage of the simulator and its inst^ictor support features ^ 

T^gk qommonflMfv Analvsy .? R«.c.,^^.y 

The task- listing matrix that was generated for the task commonall^v 
analysis presents a listing of tasks that are trained on the ATDs inv^Mea^S 
and those incorporated in the four systems with advanced ISFs ms ifSix 
has been included in Appendix E of the ISF Guidelines. 

the \rrma? mg^.'^n^orraV-^oceTr^'^ri^^^^^^^^ 

r/conr^ this L„.on'aliVis=?enfLt^;sTht Xll 

MAJCOMs. pis is not surprising, since these types of tasks reflect Ihl 

thi'studen/ '^'^^ philosophy, wl^ich includes ensurt^ tSt 

aircraft la c ^ f''^ understanding of procedures and limitations of the 
abllltv %r '^^"•^"^trate this knowledge, as well as the motor skJll 

Zll'Vo the?i?,\%?irhr ^'"""^ ""'^'^ ^ "ndit^o'is 

The task-listing matrix indicates that conversion training-task require 
ments encompass all UPT training- task requirements In conv^s^on tr«?nl 
It stit'^w'a -^tS the new'systems and uUlLrthirto"So?; 

saLty ^f m^ht an'/ I^"' T^'^^ ^" ^^'^-^y emphasis' is^ 

sarecy ot flight and on the student's abilitv to ^nft^l^ *.u 

»»o„g training tasks „a.' not %b%:led Sote" ta:S°'\o™"'r 
applicable to all major oparational commands. Such mission-reUted tasks lit 

28 



ERIC 



encompassed In Air -to -Ground Attack and Electronic Warfare. These tasks as 
taught in conversion training, provide the basic foundation for continuation 
training in the operational squadrons. The AFTS^ for the A-7D and F-4E is 
the only system designed to monftor tactical training. These systems provide 
air-to-air and air-to-ground tr-i, - Lng that could meet the needs of conversion 
training in these areas. 

Nearly all conversion traiaxng tasks in the first four task categories 
(normal flight, normal procedures, emergency flight, and emergency procedures) 
have already been incorporated into the four systems. Xaska wnich have not 
been identified as ones monitored by an ISS wex^ not done so by design. The 
ISS systems for the F-14, B-52 ARPTT, and C-5A could be easily modified to 
monitor any additional conversion training- task requirements. 

Tnhi^T-pfll TSS Analysis Results 

This section describes the capabilities that allow systems to monitor, 
compute performance u asures and score, .and trigger other instructional support 
actions. Refer to Appendix F for « detailed comparison of ISS capabilities 
of the four systems which featura advanced instructor support. It should be 
noted that these systems wero all modifications to existing ATDs and that the 
system design was dictated by what existed. 

Conceptual View 

A conceptual vf.ew of the ISS is presented in Figure 3. The ISS is 
viewed as a device that may monitor any variable present in the simulation 
(e g variables for flight, navigation, controls, environment) and take 
actioA when specified criteria are met (e.g., altitude - 1000 ft). The 
variable-? to be monitored and the actions to be taken are dictated by a 
stored script; also, a new script can be initiated when specified criteria are 
met In this view, the ISS is a controller of the instructional process, and 
much depends on its ability to monitor, interpret, and diagnose ongoing perfor- 
mance and to take action based on complex statements of performance conditions . 

Tffpy p eftnitlon . The concept of a task module is used in two of the 

four systems examiiied in this analysis (F-14 ISS and ARPTT); however, it has 
been applied to the entire discussion, since it provides a functional 
description that includes traii.lng relevance and some independence of specific 
methods of engineering implementation. Task modules are instructional building 
blocks that describe the training objectives at a level which has meaning to 
instructors and that can be used by the machine to monitor and control in a 
manner appropriate to the instructional objectives. Thus, if a training 
objective is to execute a standard instrument departure with malfunctions 
inserted under specified conditions and to measure flight, navigational, and 
procedural performance, all such information would be included in the task 
module definitions corresponding to the training objectives. Upon completion, 
new task modules can be referenced by the ISS until all objectives for a given 
mission are included. The task module, then, is a control file or a script 
which drives the ISS and which can be interpreted by instructional personnel 
and related to training requirements. 
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SIMULATOR 
VARIABLES 



INSTRUCTOR 
INPUTS 



MONITOR 

ACTUATION 

CRITERIA 



NO 



NO 



TASK 

MODULE 

DEFINITION 




CRITERION 
TRUE? 



START/STOP 
PERF. MEAS. 

» 

COMPUTA* 
TIONS 



COMPUTE 

PERFORMANCE 

MEASUREMENT 



TABULATE 
SCORES 



END/BEGIN NEW 
TASK MODULE 




TAKE OTHER 
ACTIONS 



CHANGE DISPLAYS 



DIAGNOSTIC MESSAGES 



CHANGE SIM SETUP 



INSERT/REMOVE 
MALFUNCTIONS 



RECORD DATA 




INSTRUCTOR 
COMMANDS 



ALTER INSTR. 
SEQUENCE 



CHANGE/ADD SCORE 



CHANGE MODE 



Figure 3> Conceptual Model for ISS Control and Processing. 
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start and Stop of Coniputatlons . An important feature of the ISS is the 
ability to recognize conditions and to start and stop ISS processes; for 
example, to start a new task module when a complex combination of events 
occurs and then to end it later and to start/stop measuring performance 
(e.g., measure average heading error when between two NAVAIDS) . This match-a- 
pattern-take-an-action characteristic can permit "smart" behavior on the part 
of the ISS, and if the ISS is truly to support the instructor and avoid 
inadvertent actions, very complex recognition may be necessary. The actuation 
criteria that: can be specified for a given ISS was therefore a fundamental 
determiner or ISS performance. It should be noted that start-and-stop condi- 
tions are often difficult to describe precisely enough for computer recog- 
nition. Therefore this remains one of the primary challenges in developing a 
"smart" ISS. 

Performance Measurement . Performance measurement is an ISF of an ISS, 
and in fact, such systems are often called Performance Measurement Systems. 
Of course, performance measurement is important for scoring and grading, but 
is also important as an adjunct to normally available simulator variables for 
the purposes of control of ISS instructor support features. 

Instructor Support Actions . An ISS may incorporate a number of ISFs 
which require direct control of the simulator and setup conditions, instructor 
console displays, insertion and removal of malfunctions, display of diagnostic 
messages, and the recording of detailed data for post-simulator use. Appendix F 
describes the comparison of instructor support actions among the four systems 
in greater detail. 

Instructor /ISS Interaction . Although an ISS derives much of its effect- 
iveness from automatic functions, it must also support an instructor in a 
flexible manner, allowing the instructor to override and re-direct its actions. 
For example, the instructor may wish to vary the sequence of task modules, skip 
a task module, or alter the conditions under which a malfunction is inserted. 
The ability for flexible interaction between instructor and ISS was examined 
during comparisons among the selected systems and is discussed in greater 
detail in Appendix F. 



Conclusions 

A view has been taken that the ISS is a programmable controller and 
a generator of information, and although the four representative ISSs vary in 
the way they are implemented, all fit the same general model. The method of 
specifying the program for the ISSs varies, but the task module concept can be 
used for initial specification for any ISS. This implies a data-driven 
system, but specification in this manner still permits a large degree of 
design freedom. The tjrpes of actuation criteria included in an ISS determined 
how "smart" the ISS can be in behaving "intelligently" in controlling instuc- 
tional events and features. The ISS can be the generator of a large amount of 
different types of information, and this capability depends on the manner in 
which performance measurement is implemented and the types of data recorded 
and displayed. Although all four systems provide a preprogrammed automatic 
mode, each provides some manner for instructor control and override, allowing 
a degree of flexibility in use of the ISS for tailored instruction. All 
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provide instructional support through the control of performance measurement 
scoring, displays, malfunctions, communications and data recording ^^r^: 
four systems provide a range of examples that characterize the current VlT 
nology and provide a basis for the generation of future specification: 

Hardware anri Software Tm plications 

softwS: zir:..iL''^i sriz^LZ rerurrL^r-xh:%ot 

lTf:Llr r^^"^ engineering. hardwlre^Tnd software'r/rspe i^ ""Z 
rLulrlen^^^'"f^ '° correlate the resulting ISS characteristics with Iht 
requirements and to note commonalities amone the %m,r r«o«- ^"'■J-'-'' wicn cne 
However it was dlff^r„1^ t-« >, t t ^ ^ ' examined. 

However, ic was difficult to break down for meaningful comparison. 

ir.■F^Jtll^r.^°'^''^,^^^^V^^ J'®''® developed as adjuncts to existing ATDs The 
Hardware and S oftware Component-^ 

associated group of characteristics are described further below 

Si£i~— ^^^^^^ 

r«ni„^ AFTS^and F-14 ISS. speech generation devices provided for the natural 



Simulation Interface . In two of the four systems ^AFTS® «,r,rf r -^an j ^ 
acquisition unit was used to examine data that^lowed be^een the s 
computer and the simulated cockpit devices. In the C-5A Ms " H^Lf n ! ° 

P i: r. in the F-U ISS. the switch unit was interposed between the simu- 
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la t Ion input /output (I/O) processor and the slmulatloty. computer 's shared 
memory. The data acquisition unit utilized on the AFTS^and C-5A PMS was 
designed to accommodate 2000 channels and burst rates of at least 750,000 
24-bit words per second. The F-14 ISS dealt with 32*bit words sent over a 
2-Mbit serial stream. The B-52 ARPTT ISS was unique in that it shared memory 
with the simulation computer. All interfaces were designed to capture and 
send infomnation on ATD parameters necessary to monitor and control the ATD on 
a non-interference basis. Tradeoff studies were performed to identify the 
means considered most cost-effective for the existing ATD configuration. 

Computational S vs tern , All four systems were implemented as stand-alone 
systems that utilized their own processors and storage. Design tradeoff 
studies performed on the C-5 ATD and ARPTT ATD considered the possibility of 
utilizing existing spare capacity within thrs ATD host, but it was decided that 
the system could be best developed by using additional processors to minimize 
impacts on the existing ATD, All computers were commercially available. 
Their computing capacity is expressed in terms of Data General Nova instruction 
execution and FORTRAN Whetstone benchmarks as points of reference in Table 3, 
Three of the four systems distributed the computing task further, utilizing 
more than one minicomputer. 

For the most part, the processors chosen to perform the ISS control were 
minicomputers. When the first prototype AFTS® was developed, one of the 
goals was to show that the system could be built using minicomputer technology, 
rather than mainframes. Proven effective in AFTS®, similar (but of the 
technology commercially available at the time of their procurement) mini- 
computers were applied to the F-14 ISS and the C-5A PMS, 

The Perkin- Elmer 32/D was a departure from the above in that the ISS 
was procured as part of an upgrade of tiie entire ATD to meet a 15 -year life 
cycle. The 32/D was a cost-effective, vendor -supported upgrade for the 
existing ATD; the selection of an additional 32/D and a shared memory interface 
was a good logistical choice for the ARPTT ISS, as shown in the life cycle cost 
study performed as part of the ARPTT upgrade study, 

gys tem Performance , System performance takes into account whether the 
capacity of the computational system adequately supported the functional 
demands upon the system,, such as monitoring cycle time, number of simulation 
variables, expected task module concurrency, and the software architecture. 

In all four systems graphics computation was to some extent off-loaded 
onto the graphics device. In the F-14 ISS and C-5A PMS systems, a dedicated 
minicomputer was allocated for graphics processing. It should be noted that 
the processor used for graphics on the C-5A PMS proved to be inadequate due to 
demands on its capability to load files off disk. 

The F-14 ISS computer performed adequately for a single, off-line or 
real-time activity; the disk file management demands of the software archi- 
tecture precluded effective concurrent activity. The C-5A PMS computer 
effectively handled all activity except for the monitoring of momentary 
switches. An attempt at increasing the cycle time to 200 milliseconds from 
800 milliseconds was thwarted by the lack of sufficient spare computing power. 



33 



CHARACTERISTIC 
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Tablo 3. Hardware and Softwara Characteristics 
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8-82 ARPTT ISS F-H ISS 


C-SA PMS 


Stations 


2 


4 


2 


3 


Location 


OflboJrd 


Lelt and right seat onboard (replaced 


onboard 


'Instructor pilot onboard station 






Electro-optical Visual System displays, 




(station mod.) 






iigni iiii 










9 nlltvi>rH Imvi In nimfit nl hr#uiMlc 

A UlltAMIU imiO III fttmMi Sit ^IWiWJt 




Instructor llight engineer 






operator console) 




onboard station 










1 onboard 


Functions 


1 lor monitorino and replay 


Ontx>ard - all but debriel 


1 lor primary control, monitoring 


Real*tifr)e mission monitoriryg and 




1 lor just replay 


1 ollboard - all but debriel 


1 lor briel/debriel, setup, and 


control onboard 






1 nlltvtarH all hiil ninfinM 

1 UIIUUwlU •!! I |*J||IIII^ 


fill nriil 


Startup, generation* and data analysis 










ollboard 


Concurrency 


Runtime, & replay or hardcopy 


Runtime and any or>e ollline 


Runtime & btiel/debriel & hardcopy 


Runtime and debriel reports hardcopy 










(graphics hardcopy directly oil IP 










display) 


Input Devices 


Keyboard end function keys 


Touch pad devke 


Touch pad over 1 9" monochrome 


Special lunction keyboard 




Speech input 




CRT 


nvvDoarin oiiKXMra 


Output Devices 


CRT displays, synthetic voice 


Onboard 9" mooochrornc CRTs 




Instructor pilot display with graphics 




Printer/plotter 


onboard 19" color CRTs 


CRTs 


hardcopy unit 








> 1 iniei/pioiiBi 


9" monochrome CRT lor IFE, 15" 








Synthetic voice 


monochrome CRT. 2 CRT terminals 










Line printer 


Simutation Interface 


Switch unit diverts data How 


128 Kb shared memory with sIm 


Switch unit diverts data How 


Switch unit diverts data How between 




between sIm computer and sIm 


computer 


between I/O coniputer ar>d ATD's 


sim computer and cockpit. Also inter* 




cockpit. Concern lor inlormation 




shared memory. 


cepts IP key inputs/responses between 




How rate» 






IP station and sim computer. 


>>n)putationat System 


Data Genera] Nova 3 


Perkin Elmer 6/320 


Data General S-2&0 


Data General &130 




Speech processor (SUS-Nova 3) 


(ISS and graphics control) 


(ISS control) 


Data General Nova 4S 




Graphics ptocessor in display device 




Data General &130 (for graphics 


(display and communications) 








control) 




Computing Capacity 










Data General Nova 










instructions 


1.16 




S250- 1 23 


S-130 - 1.23 


MIPS 






S-130 - l.l^i 


Nova 4S - 1.8 


KWhets 




901 


S 250-3t^' imate) 


S-130 ^ 240 


(FORTRAN) 






s-no 


Nova 4 - 185 


Stora^ 


96 K main memory (SUS - 32K) 


896 K main memory 


fr2&0 ' 1.0 \^ 


S130- 512 K 


(in bytes) 






fr130 - 384 IC 


Nova 4S 64 K. 




10 M disk 


80 M disk 


96 M disk 


10 M disk eKh 








10 M disk 
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CHAnACTERfUIC 



AHPTT IS5 



SYSTEMS 



F-14|SS 



C 5A l>MS 



System Performance 

Monitoring Cycle 
Variables 



Software Aichitecture 



Strmjlalion Variable 
Processing 

Software (Source Lines of 
Code (SLOCH 

Control 



Measurement 



Brief/Oebrief 
Tutorial 

Training Managennent 
Other 



Good performance with ipare 
capacity, ilowing of replay 
concurrent with runtin)e 

200 ms 
800 ms 

10-30 for a given task 



Task Module Concurrency N/A 



Task modulei embodied in coda 
Parameteri for scoring, airfield, arid 
curriculum rede fin able via editors 
Software scheduler brings in blocks 
of code and data for mission segtmnts 
(Airfield Proc«durw (APRK GaR/GAT, 
AAI) bated on curf iculum deMnition 
and algorithm for adaptive curriculum 

Values of interest for tha activff 
task ar0 evaluated 



Executive control and div>^ 
drivers - &8,000 5L0C* 
Display - 60,000 SLOC* 
Airto>air, ground attack, end 
airfield procedures control, and 
display - 26,600 SLOC' 
Editors for curricuium. airfiefd, 
and scoring - 26,000 SLOC* 

Geometry, voice replay - 6800 
SLOC 

None 

Student records handling - 4000 
SLOC 

Systems confidence tests, speech 

recognition utilities - 6O0O SLOC 

Diagnostics 

Land mass calibration 



Good performance during concurrent 
activity, slowing of simulator variable 
proceising 

100 ms 

664 (about hall monitored) 
2 



Task modules embodied in data read 
in via dtik files 

Monitoring separated Into continuoui 
measurement dorte iyndironoi^*Jy and 
procedural events done asynchronously. 
Task mo(ftjles Identified for million 
are read in and acted upon 
Editor for module definition 

Values of interest evaluated whenever 
required 



Real-time instructor control, menu 
management - 7200 SLOG 
Oiiplay - 4000 SLOC 

Procedures/event monitoring^ and 
data recording - 2700 SLOC 
Exercise definition - lOOO SLOC 
Interactive task module editor - 
340O SLOC 

Logon and debrief 2000 SLOC 
(graphic replay) 

None 

Mission data analysis - 810 SLOC 
Registration - 1000 SLOC 

Picture generation, menu creator - 

1600 SLOC 

Test driver — large 



Good performance for lingle activity, 
S-250 not adequate for concurrent 
activity control, 5-130 adequate 
200 ms 

166 (about 280 items) 
4 

Task modules embodied In data read 
in via disk files 

Monitoring done by asynchronoui 
event detection (event detector is 
synchronous), messages are passed 
on to activate response to the event 
as defined within the task modules 
Macro aisembler used for module 
definition 

All simulation variables evaluated 
syr^chronouily 



Real-time instructor control menu 
management 5670 SLOC 
Diiplay - 1560 SLOC 

Procedures monitoring, performance 
measurement, scoring - 1S60 SLOC 
Exercise definition - 1020 SLOC 



Logon and debrief - 1800 SLOC 
(graphic replay) 

Tutorial on ISS - 1020 SLOC 



Controller modeli » 1080 SLOC 
Teit driver 



Good monitoring performance except foi 
momentary switches. Nova 4S not adequ 
for diiplay procusiing 

600 mi 

(slow for momentary switches) 
About 2400 

10 <8 checkliits t navigational profile * 
monitoring parameters) 

Task modules represented ai a navigatior 
profile and mission segmcnti read in as 
data from one precompiled disk file, all 
values converted. Monitoring leparated 
into continuous measurement and pro- 
cedural event I monitoring, all done 
synchronouily. 

Language provided for module definition 

All limulation variables evaluated 
tyr)Chronouily 



Real-time instructor control 
12,600 SLOC 
Oiiplay - 6O0O SLOC 



Mission profile generation 
SLOC 



18,565 



PMS research utilities - scoring, reportin 
data retrieval/analysis - 77 tO SLOC 



Confidence program, diagnostics 
10,716 SLOC* 



* Includes significant amounts of assembly language source lines of code 
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co«^.!^ / performance capacity of the B-52 ARPTT ISS 32/D allowed all ISS 
control and graphics to be hosted in one computer. AFTS® arid A^Sr flit 
adequately given the required functionality, but with little .n^r!! P^'^f? 
and memory capacity. iittle spare computing 

except the hVSS^, via runtime control actions. It is of Interest to 
PmI in which a uZ^^ ^as d;,eloped "peci.lly in c-SA 

code 4V»aiS M'/ues Tr^t" '0^0^.:^ '"^^1." 

code SUIcheTler'' f ® ^^""'^ (synchronous) execution cycles with 

coae being scheduled and executed on one of two available cycles 200 or flOO 
milliseconds. The P-14 ISS and B-S? arptt tcc u j "uic i-yci.es, zuu or 800 

student actions could be acted u'^^^^^^^^ "^-^^^ or 

step, out-of. tolerance et,. cZl^^ trigger (start, stop, procedural 

adjS^ted to\a:l^'Z:\:^:iL::::^l ^^"^^^^^ ^^^^^-^^^ priorities cou^ be 

b»dg el°a^rTtra?eT fX"for P"-""-"^ °f - ISS can address needs and 

uuugecary tradeoffs for a minimal to an all-encompassine T<?<? Vnnf. tcc 

torl4; SceS^S'roY r"fo«.nce Measurement. Procedures Monl- 

Graphics Repu/^"""'^ " ="^''""8 "lUtles, Hardcopy/Printout, Re-ote 
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4. Tutorial Tutorial 

5. Training Management Data Storage and Analysis 

At a minimum, an ISS requires display and control functions that allow 
the control of the ATD and the real-time monitoring of the student. Given 
this core capability measurement, then brief /debrief, tutorial, and training 
management can be added as required to procure a system that meets the needs of 
the user community. Table 3 presents information on software components imple- 
mented for the four systems, broken down by functional area to give a frame of 
reference for future procurements. 

The software for each of the systems was written using combinations 
of assembly language and available high Icjvel languages, such as FORTRAN and 
Data General Corporation's ALGOL (DG/L) . Note that source lines of code 
(SLOG) numbers identified in Table 3 include extensive assembly language 
lines, resulting in much higher numbers for the AFTS®. Operating systems and 
utilities commercially available for the hardware were adopted for both the 
development and runtime operation of the systems. All operating systems were 
multitasking, able to keep track of more than one system activity at a given 
time. Also, the F-14 ISS took advantage of its multiprogramming operating 
system's messaging capability to implement asynchronous system control. 

Conunon Characteristics 

Given the collected data just discussed, major characteristics common 
among the four ISS systems lead to the definition of a stand-alone system for 
ISSs which can be added to existing ATDs. This stand-alone, modular ISS has 
the following features: 

1. All ISS processing is isolated from the simulation processing. 

2. Data are passively shared with the simulator. 

3. The ISS can be added with minimal modification to an existing ATD, 

4. The system does not require large, i.e., mainframe, computers. 

5. The system attempts to provide user stations that are tailored for 
effective instructor use. 

6. The training tasks are specifiable and updatable without requiring 
system reprogramming. 

7. Software is functionally modularized. 

8. Commercially available software and hardware are utilized to the 
greatest extent possible. 

Hardware and its accompanying software development environment can be spec- 
ifically selected to support the ISS functions or can be expanded in function- 
ality without impacting the simulation. Additionally, adverse impacts on 
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Ill aSS ™ ^/ development can be minimized. For example 

P ?A ^Ic ^^^^^ existing F-4E and A-? ATDs in about a week. The ms®' 

F-14 ISS. and C-SA PMS could also be switched completely off as necessaS t; 
allow use of only the pre-existing ATD. f J' necessary co 

T«!«! n!!;^ synchronous rate required for simulation processing is not required for 
ISS processing For example, the F-14 ISS was attached to the 2F95 ATD whicS 
had a cycle time of 20 Hz. The F-U ISS successfully performed all dirplav 
and "onitoring functions using a 5-Hz simulation variable monitoring ScJe 
coupled with asynchronous processing of other activity. The B-52 ARPTT ISS 
designed in a similar fashion, allowing for monitoring of events ^f inLre^? 
Sr^a'te"' °f-ntinuous flight variables, and updating che dLpUyVat " lO: 
Hz rate The application of a small, synchronously execut in/ kernel o£ 
software dedicated to the detection of eventi of interest couirpossibTy havf 
alleviated the problem of detecting momentary switches in C-5A PMS. 

Software modularity allows for addition of functionality in phases For 
fXlit' capability was added to the F-U ISS^fter all othe^ 

functionality was completed. The B-52 ARPTT ISS was delivered in two phase" 
the first phase provided monitoring and display functions whn» P^^^es 
provided measurement and training maLgement fCc^ifr"^'ISs'w^^^?4atl::;? 
o1lL':^'cMs:'^ '^"^ ^'^^^ ^'^^ "^"^-^ lr°- ^.anaSol 

The above constitutes a baseline description of what falrlv hi 
commonalities may be carried forth into a generic archittture for'^an ISS Th; 

oHwi'''" ^""'^ ""^^^ P-»= functional rTquiremeSs 

of these systems, to a lower level of implementation detail. -^quirements 

tected°S "^Q-^ ^^l! ^^r^"^ "present a progression from AFTS®, which was archl- 
xltT J\ i , °" studies performed since 1969). to C-5A PMS and F-14 

I'frind I9T2" ZTst pS.''.?; r/'! '''' ^^^^^ wasTes^g'ed 11 

I t 1 ; indeed reuse pieces of the AFTS® execv'tive 

arc^?^ T"' ""^^ transferred the F-14 ISS software 

architectural concepts to a different training problem and hardware "It^ 
ISdlt^o""f^'" successfully accomplished but not without a great deaJ of 

additional customization and development. This aeain 1«5 ^Jrilr^r 1 \.u 
funding required to build the production AFTS® basfd on work done ^n th^ 
prototype AFTS® (see the following discussion on cost factors) 

The development of these systems has capitalized on previous lessons 



requirements of each of the procurements! 
Cost Factorfi 



elements fcro./ ^>f T accurately separated into the same cost 

dJ'jrjl t f systems, since the work breakdown structure was 

different in every case. The significant results were in the area of relative 
contractor development and procurement costs and identification of colt 
re'viTwed' " " '"^^ ^'""^ ^^^^ contractor cos?s ^^re 
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The relative costs for the AFTS^ 
Table 4 as points of reference. Note 
not reflect the costs associated with 
which preceded. 



production and C-5A PMS are shown in 
that the C-5A PMS and AFTS® costs do 
the research and development studies 



yable 4 . Relative Development and Production Costs 



Svsteros 



Cost Category 



' AFTS® C-5A PMS 

A-7D: 1 proto, 4 production 1 research/develop- 
F-4e': 1 proto, 15 production roent system 



Management 


7 % 


9 % 


Software Engineering 


8 % 


47 % 


Hardware Engineering 

Manufacturing/ 
Procurement 


6 % 
39 % 


32 % 

(engineering and 
procurement) 


per unit 


2 % 




Installation 


5 % 


4 % 


Provisioning 


18 % 


N/A 


Data 


13 % 


7 % 


Quality Assurance 


2 % 


N/A 


Final Contract $ 


9,690,928 


1,237,252 


Reference Year 


1978 


1980 


Adjusted Amount in 1984 $ 


17,637,489 


1,670,290 
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software p,oiules Implemented In C-5A PMS *d F-U ,« °* different 

of code figures are elvan rr. ^J™,.^. I I I ^'liaMd source lines 

the softwale de "lVm^;\%?£St ''"Note ?w ".L^'f""" *° ""^""-"^ 
mlJt of assembler (both iss") FOrSa^ 5a pJ/T"""'' ^ 
comparable costs are'not a 4ualle'?'fSe 'oth* Zl^J^" 



Iakle_^. Relative Software Costs 




Software Category 



Top Level Analysis/Design 
Display 

Real-Time Monitoring/Control 
Off-line Analysis/Debrief 
Scenario Generation 



C-5A PMS 
SLOC/% Cost % 



N/A 

6,000 13 
13,000 28 

8,000 17 
19,000 41 



14 
4 

50 
3 

28 



F-14 ISS 
SLOC/% Cost % 



N/A 

2,000 14 
9,000 64 
3,000 21 
N/A 



19 
12 

62 
6 

N/A 



of a co'SolToperX\raddil^^^^^^^^ "'l^ ^^^^ ^he elimination 

for a 15-year 'life cycL wtr:^'i^r'Lt^rs'"^;^^^^^^ supportable 
objective of the ARPTT uoerade atudv *-« „ Again, these costs reflect the 

a 15.year life cycle "PS"'*^ ^" existing prototype ATD for 

Selection of r:..fH^itnes Pnr-n.o- 

OuldeU„e7Y„i^r„i" - be°:aL"b"thf MrXrcf T"' °* 
selection of a format were that Itnv^frf. ! ' '"eS^^'^d criteria for 

tlon and that It be eaTy to "e anr«L«nce ""^"'■'-l- l-forma- 

suppo«%apiwuri:s"L'^\Tl 'L°'rr *° "^■^^ instructor 
euldes. For this re.'sor the narrative Sf"^ l"*"" specification 
It can present a deeoer lav«r^? f 7 fT recommended because 

selected The loraVrkinrerouo^lT. subse,uently 
layout for the vU^L\ X^nyioTofThe^'S.lSunrs*' *° Mappln^ 
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It was felt that a single guidelines document to be used by the spectrum 
of individuals involved In specifying ATDs, rather than several volumes geared 
to separate audiences, would promote a greater degree of mutual understanding 
of ATD requirement specifications. To facilitate information access, tabs and 
an index have been included. 

Computer -readable diskettes containing the Guidelines were provided to 
SimSPO upon submittal of the final deliverable of the ISF Guidelines. With 
regular on-line update, the Guidelines can serve as a repository for lessons 
learned to benefit all users. 

yhe Final Product ISF Guidelines 

The ISF Guidelines were developed as a result of all of the efforts des- 
cribed. The review of ATD acquisition/specification process and instructor 
support feiiture analysis provided valuable insight into user requirements. 
The internal ISS comparison and the examination of hardware/software impli- 
cations provided important lessons learned about the development and utilization 
of state-of-the-art systems. It is hoped that use of these Guidelines, 
written and formatted with the readers in mind, will lead to better written 
specifications for ISS capabilities of future aircrew training devices. 
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IV. CONCLUSIONS AND RECOMMENDATIONS 

Front-end analysis for both student and Instruc tor reautrements Is needad 

To ensure instructional system performance fully supportive of the specific 
requirements, a comprehensive front-end analysis of instructor requirements, as 
well as student training tasks, must be completed. Requirements for 
instructional systems which are copied from seemingly similar recent procure- 
ments may av. best be approximations of what is actually L'equired. Efforts 
committed to proper planning, analysis, and specification of the instructional 
system will prevent specifications that do not meet user needs. 

Instructor training Is needed 

There have been several documented studies, including the one conducted 
during Phase II of this project, which point out that instructors are not 
properly trained in the use of ATDs , especially in the area of instructional 
features- A well designed instructional system for the AID greatly improves 
the potential for simulator training in many ways that are not available when 
using Che actual aircraft. This potential will not be realized (and the 
features will be ignored) if minimal time has been allocated for training the 
instructor in the use of the instructor support features, or if the instructor 
infrequently uses the simulator. 

The ISF Guidelines have been carefully written in user- specif ic terms to 
provide operational definitions and lessons learned. These guidelines would 
be a valuable tool during initial formal training to introduce the instructor 
to instructional features and their intended use. 

Ongoing instructor effectiveness would also benefit from automated 
instructor support system capabilities specific to a particular ATD if they 
were batt^ir understood with respect to specific training objectives. Therefore, 
the design should include provisions for built-in instructor tutorials and help 
features specific to the device. 

Task modules provide opeT Tat-^iy.nT^al, data requ ired for the instructional systen^ 
to support usey needs 

In order to provide user- oriented and meaningful "on-line" support, an 
instructional system must "know" where the student is in a training mission 
and to take preprogrammed action under appropriate circmstances . The £a§Is 
module (user-oriented building blocks which have been transformed into software 
data files) has evolved as a central concept in this regard, containing 
the logic, performance algorithms and criteria, displays, data recording 
and other actions to be taken. The task module provides a means for commun- 
ication between the opferati.r:nal personnel and the contractor: it provides 
the means for operatiori:il personnel to specify precisely to what extent the 
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ISS is going to support the achievement of each ins^r„r^^ v . 

provides the system develooera «^^-h instructional objective; and it 

achieve efficient and eSecri^e tracing controlling the system to 

^r^^rl'^B^^lT^^^^^^^ system that .he 

may be directly correlated with a list L tlti T. ^ T "^"^ objectives 
which may then be used as Part of the des^L cr\\eri^^^^^ described above) 

Ihe l,p?tpict:j.Qna1 ■sv<;rem sho..^r^ ^ent a^ ^.^.r.^ 

.6.inTs.rr!^'Z,: fnrjtl^^l^^^ ^ '^"S Co be 

in the syllabus, information for each flight m^n for each mission profile 
With performance measurement criSria ' and def^^^^^ procedure, together 
generated display Cincludlncr nn,M definitions of each computer- 

These data mu^st^i kept c^rre^^^^ departure templates", 

flight procedures; otherwlsT the inst^^ M^" ''"'^'"^"^ requirements and 
ineffective in the ar;as of ^h^e ^^^^^^ system becomes unusable or 

learned from the systems that Lvf beef^tesSd^?: o^U^tio^l^rriLi^e^^ 

e^o.oT'^Zr^K'^^^^^^^ include provision for 

pushed using the task module coLJpt Tdat^ba^^ 

dynamically changing da.a describ^rabov/t^ tin^^^^^^^^ architecture) . The 
and contained in a df^tabase The flip? ^f^^^'^'^^bfd into computer files 

software while the system is in deration. ''"^ ' database -driven 

acco.;?ired\y^°roSarTutili^ ^^^^V^^ ^^^'^ 

specific software skiUs aS expertis^ t:W% P"^^"""^ dependence on system 
This update could then be accomp^^^^^^^^^ time-consuJ^ing. 
any operational objective/proceXre change ''^ ^^^'^^^ 

Uv^;^ pf .T,t o wttoTi sh ould dnpnnd on th. ^r.-rn.ne ob^P■c■^iv. 

complin: L^'atL^of^aVr^i^nr^^^^^^^^ TncTudin^ t^^^ "^^^^'^ °^ ^"-^^^"^ 

required inputs during a session including the preprograming of all 

effective when the iL^ructor's ' prLnce ^s' olimtrn'^"'"' T 

co-if-ori™^^^ 

tha student's needs instructor's capability to tailor the session I'o 
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The design of the instructional system should be especially sensitive to 
the automated control features and provide the levels required by the training 
objectives specific to that device. Providing varying levels of automation 
that can be selected by the instructor as part of the exercise definition 
process provides the flexibility to cover a broad spectrum of training 
objectives of many ATDs. 

Automated performance measur pmpyit; should b e de slp ;ned as an aid to the instructor . 

In the past, performance measurement was too inflexible and rudimentary to 
provide meaningful feedback. For example, some performance measurement 
systems provide single aircraft parameters that have no meaning when presented 
in isolation. Such parameters should be presented in combination with other 
parameters and student actions to present a more complete, and therefore 
more accurate, evaluation of the student's performance on an objective. In 
other cases, performance measurement systems have provided a single score for 
a maneuver without any supporting data. This type of measurement is also 
usually ignored by instructors and students because it provides no information 
with respect to how the score was obtained. 

State-of-the-art, automated, performance measurement technology evaluates 
performance by objective, taking into account all of the actions required to 
perform the maneuver. A review of che measurement of each action, along with 
the evaluation criteria, should be liiade available such that the student and the 
instructor can determine both whether a criterion was met and how it was 
performed. 

Automated performance measurement should be designed to aid, not replace, 
the instructor in his evaluation of student performance. 

^equlre^nents should not be overspecif ied 

The specification of instructional features should be based on functio- 
nality rather than technology. Within the basic instructional system archi- 
tecture, use of available technology is to be encouraged by not over-specifying 
hardware or software components. Technology changes daily: computer hardware 
costs are decreasing as performance continually increases; new input and display 
devices are continually being introduced; software technology is on the verge 
of making major adveaces with Ada as a programming language; and artificial 
intelligence and other approaches are emerging. Therefore, specification of 
functionality and performance from a user's perspective is imperative. This 
will allow contractor latitude in providing SimSPO with a spectrum of 
alternatives that will maximize the application of current technological 
advances and current standards . 

f!^ ^ Guidelines should be maintained as a dynamic document 

The product of this project, the ISP Guidelines, is intended to effectively 
transition lessons learned and proven technology from advanced instructional 
systems into the operational training environment. As rapid advances in 
weapon systems technology occur, progress in instructional systems technology 
is expected. With the introduction of sot tware- driven cockpits, cactics and 
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?rc?et"%nrt^t?A '^^^ *=hreats and adversaries cannot be pre- 

aiccea. Instructional systems technoloev will mee^ t->,^a ^K«n ;\ 

8«ldaUjjes, too. should accommodate those changes with continued Spd^te 
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APPENDIX A 
TRAINING SITES VISITED 



Training 
Device 




Dat^ 


AFTS A-7D 


Davis -Hon than AFB 


12/7/84 


A- 10 


Davis -Hon than AFB 


12/7/84, 1/30/85 


ARPTT ISS 


Castle AFB 


1/15/85 


B-52 


Castle AFB 


1/15/85 


C-5A PMS 


Altus AFB 


12/12/84 


C-130 


Little Rock AFB 


12/11/84 


F-15 


Luke AFB 


11/15/84, 1/29/85 


F-16 


Luke AFB 


11/15/84, 1/29/85 


T-37 (T-50) 


Williams AFB 


11/13/84 


T-38 (T-51) 


Williams AFB 


11/13/84 



53 

63 



APPENDIX B 



INTERVIEW GUIDE 



This interview guide was used merely to euide the IntervlPT^ ho 
the issues and not to collect data. BecaLe tL number of i^str^^^^^^^ 

the nex't .^"^"^ "y^^^" "^'^^^^ 3 ignif icantly f ro^ one ATO to 

the next, this form was used as a general euide The rp.>,.lfcir„r a^^I u 

su.^rtz,d tn section III. KeJts, Inri«r Xpo" P^/tutrA^^Jy^L'" 
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INTERVIEW GUIDE 
SYSTEM LEVEL CONSIDERATIONS 



Contact: Phone: i ) 

Position: ~ — . ^ 

Site: ATD: 

1. What are the training objectives? 

2. What are the specific tasks? 

3. What is the performance criteria? 

4. Can you provide any grade sheets or records of accomplishnent? 

5. Does the simulator capabilities match the stated requirements? 

a. Does it train to the objectives? yes no 

b. If not, what the are shrettV. s? Is it because the simulator 
can't do it, or it* it becaua..? : s not used? 

6. Are there advanced instructor support features? 
If yes, do they accomplish the intended purpose? 
If not, why not? 

Yes No 

Scenario Generation 

Preprogrammed Missions , 

ISS CONTROL FUNCTIONS 

Simulation Variables Control 

Malfunction Control 

Repositioning 

Display Control 
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Yes No 

INSTRUCTOR CONTROL OPTIONS 
Display 

Scenario Changes 
Mode Changes 
Tutorial 

Algorithms /Assessments 
Grading Criteria 

Data Storage & Analysis 

Brief 
Del : lef 

Does the course syllabus match what is actually taught? 
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INTERVIEW GUIDE 
TASK LEVEL CONSIDERATIONS 

ATTEMPT TO AUGMENT OUR COLLECTION OF REPORTS 

Task listing for Simulator Training 

Material for the training of instructors, Instructor Guides 
Stan/Eval documents 
Student guides , handouts 

OBSERVE SOME TRAINING IN EACH SIMULATOR 

PERFORMANCE MEASUREMENT DISCUSSIONS 

In the context of the following matrix (as appropriate to the training) 



! ! FLIGHT ! PROCEDURES ! 





! 


! I 


! NORMAL 


I 


I I 




! 


! ! 




! 


! ! 


! EMERGENCY 


! 


I I 




! 


! ! 




! 


I I 


! TACTICAL 


! 


I I 




I 


! ! 



Walkthrough a selected task in each category 

1. Looking at fine-grained performance? End results? 

2. Comaon errors? 

3. Standards of performance? 

4. Scoring/grading criteria? Forms? 

5. vrhat is included in the debrief of the task? 

6. Accessibility of Performance Measurement information 
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Listen for: 

1. Instructor worklcc-d level 

2. Information not detectable by machine 

3. Difficult-to-define or to-evaluate task factors 

4. Uncertain, complex task sequencing 

5. Out-of- the- ordinary performance measurement algorithms 
COLLECT CONTACTS FOR FOLLOWUP IF NEEDED 
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APPENDIX C 



COURSE DOCUHENTATION 

A- 10 

lOS Manual, Upgrade Training Course (not dated) 
Flight Objectives Pamphlet (8/84) 
TAG Syllabus (8/84) 
Gradesheet 

B'52 

Training Program WST Courr.ebook (not dated) 

Console Familiarization Course (1984) 

WST OIS Console Operations Guide Vol. I and II (8/84) 

WST DDS Console Operations Guide (8/84) 

Test Option 5 Scenario Description (not dated) 

C 130 

Pilot Study Guide Part I, Pilot Initial Qualification Course (10/82) 
Student Study Guide Part II, Tactical Mission Qualification Training 
(12/82) 

Instructor Guide Part II, Pilot Requalification/Upgrade Course (1/83) 
Instructor Guide, Navigator Mission Qualification (12/82) 
Mission Profiles I - V (not dated) 

Nulimeyer, R.T., and Rockway, M.R. Effectiveness of Weapon System 
Trainers for Tactical Aircrew Training * In Proceedings of Inter- 
service/Industry Training Equipment Conference and Exhibition, 
v^ctober 1984. 

Pt . : . ,i Preliminary Simulator Instructor Guide for Tactical Mission 

Qualification Training (12/82) 
Flight Simulator Operating Instructions (10/82) 

C-141 

SiM/C?T Instructor Guide, Pilot Initial Q-oalif ication Course (11/82) 
Flight Instructor Guide, Pilot Initial Qualification Course (1/81) 
Instructor Guide, Pilot Airdrop Qualif icvtion Course (11/83) 
Instructor Malfunction Guide, FligT Knginear Initial Qualification 
Course (3/34) 

Flight Instructot Guide, Navigator Airdrop Mission Qualification Course 
(3/83) 

Task and Objectives Document, Loadmaster Airdrop Qualification Course 
(10/81) 
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F-15 

Operational Training Course (10/81) 

Simulator Instructor Pilot Upgrade Procedures (7/83) 

Instructor Operator Guide, F-15 Simulator (7/83) 

P* 16 

Basic Operational Training Course (1/84) 
Wordstar Lesson Plans (1984) 
Gradesheets 

Instructor Handbook (3/82) 



KC-135 



Pilot WST Coursebook (1/84) 
Navigator WST Coursebook (1/84) 



T-37 



Instrument Program (3/83 and 9/84) 

Syllabus of Instruction for Undergraduate Pilot Training (T-37/T-38> f8/83^ 

T-50 IPS Mission Guide (3/83) ^ \ /^^f 

T-38 

Syllabus of Instruction for Undergraduate Pilot Training (T-37/T-38) (8/83) 
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APPENDIX D 



SYSTEMS WITH ADVANCED INSTRUCTOR SUPPORT CAPABILITIES 

Autotnated Flj.ght Training System (AFTS^. Developed In the late 1970 's, 
the Automated Flight Training System (AFTS®) was installed on the Air Force 
P-4E and A-7D flight simulators for use by the Tactical Air Command (TAG) and 
the Air National Guard, This system provides the capability to run preplanned 
automated training sequences without interfering with existing trainer perfor- 
mance and allows a set of training courses to be run through with minimal 
instructor intervention. The AFTS® s ares the aircrew on task performance, 
and on satisfactory score achievement; the system automatically advances the 
student to more difficult tasks. This automated adaptive training is provided 
for the following exercls*is: instrument maneuvers, instrument penetration/ 
approaches 9 Instrument departures, radar navigation, normal and emergency 
procedures, ground attack radar, air-to-air Intercept, and weapon scoring. 

F-.1A Instructor Su pport. System (I SS) . The F-14 ISS was an R&D prototype 
developed in 1981 which "strapped on" to the F-14 Operational Flight Trainer 
at Hiramar Naval Air Station. Tiiis system was designed to provide state-of- 
the-art instructor support functions in the areas of procedural monitoring, 
performance measurement, briefing, debriefing, graphics replay, record keeping, 
and instructor training via a built-in tutorial. In addition, this system 
provides ins true tor -oriented simulation control according to the training 
objective. The ISS was developed and tested on site in the operational 
environment to provide direct user feedback. 

C-^A Perfo rmance Measurement Svs1;^t^ (VHS) . Developed in 1982, the C-5A 
PMS was an R&D prototype developed to '•strap on" the C-5A flight simulator. It 
provided such additional training capabilities as preprogrammed mission 
scenario design, real-time aircrew performance measurement and instructor 
feedback, and post-mission data retrieval and analysis. Various levels of 
statistical performance data were generated and recorded by the C-5A PMS 
during a mission. These data could then be retrieved by research personnel at 
any time for the purpose of performing statistical analysis. 

A ortal Refu eling P art Task Trainer In&cruc tor Support System (ARPTT 
ISSi- The Air Refueling Part Task Trainer Instructor Support System (ARPTT 
ISS), installed on the ARPTT simulator at Castle AFB in 1984, is an instructor 
support system that allows the instjructor to operate the simulator with greater 
ease and more flexibility. Much of the technology used in the F-14 ISS design 
was applied In the ARPTT, including performance measurement, procedures 
monitoring, and record keeping. An additional feature provided curricultun 
managers with trainer utilization and training effectiveness data. 
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APPENDIX E 



SYSTEMS DOCUMENTATION 



APTS® 

Training Specification for AFTS for A-7D (6/77) 
Training Specification for AFTS for P-AE (7/78) 
Performance Specification for AFTS for A-7D (6/78) 
Performance Specification for AFTS for F-4E (6/78) 
Software Users Guide for AFTS for F-4E (4/79) 
Program Source Listings 

Grunzke, P. Evaluation of thp. Auto mated Ada ptive Fliph^ Trainjr^ .gyst^ogj^^ 
Air- to-Air. Intercept Perfortnance Measurement . APHRL-TR.78-Z3 . Air 
Force Human Resources Laboratory, Williams Air Force Base AZ Julv 
1978 . ' ^ 

APT Program Description, 5/72 

Automated Weapon System Trainer, 6/70 

B-52 ARPTT ISS 

Study on the Refurbishment of Aerial Refueling Part Task Trainer (ARPTT) 

to Extend its Life Expectancy - Technical Rep-ru (10/30/81) 
Functional Design Document - ARPTT ISS (11/30/84) 
Instructor Guide, B-52 Training program Pilot BPAT (not dated) 
ARPTT Training Program 5/84 
Program Source Listings 

C-5A PMS 

C-5 Course Summary Document, ptlot Initial Qualification Course (1/82) 

CPT/SIM/FLT Student Guide, Pilot Initial Qualification Course (2/81) 

CPT/SIM/FLT Instructor Guide, Pilot Initial Qualification Course (1/83) 

C-5 Pilot Master Task Listing (3/83) 

Operations Manual - PMS for the C-5A Simulator (9/83) 

System Specification (Parts I, II and III) (12/82) 

Program Source Listings 

Swink T.R., Bucler, E.A., Lankford, H.E., Miller, R.M. , Watkins, H. , and 
waag, W.L. Definition of Requi rements for a Performance_M ^^ttr;:>rn^ 
System for . C-5 AjTrcrew Mfimher.g. AFHRL-TR-78-5A. Air F^ce H^i^ 
Resources Laboratory, Williams Air Force Base, AZ. October 1978 

Waag, Wayne L. and Hubbard, David C. The Measurement of C-5A Performance 
In Proceedings of Psychology DOD Symposium, U.S. Air Force Academy.' 
April 1984. ^ 

F-14 ISS 

F-14 Instructional Support System (ISS) Final Technical Report (6/30/82) 

F-14 ISS Operational Design (not dated) 

F-14 ISS System Development Notebook Vol. I (not dated) 

Program Source Listings 

Semple, c.A. , Vreuls, D., Cotton, J.C. , Durfee, D.R., Hooks, J.T. , and 
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Butler, E.A. The Functional Design of an Autoniatefl Instructlooal Supe grt 
System for Operational Flight Trainers. NAVTRAEQUIPCEN 76-C-0096-1. 
Naval Training Equipment Center, Orlando, FL. January 1979. 

Osborne, S.R. , Semple, C.A. , and Obermayer, R.W. Three Revl.evs of -£he 
Instructional Support System (ISS^ Concent . NAVTRAEQUIPCEN 
81-C-0081-1. Naval Training Equipment Center, Orlando, FL. March 
1983. 

Bosworth, L.K. , Kryway, J.T. , and Seidensticker , S.S. F-14 Instructional 
Su pport System flSS ) Wea pons. System Tra ining (VIST) . NAVTRAEQUIPCEN 
80-C-OO56-1. Naval Training Equipment Center, Orlando, FL. March 
1981. 
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APPENDIX F 
INTERNAL ISS COMPARISON 



This appendix presents a detailed comparison of internal ISS features 
among the four systems (AFTS® , C-5a PMS. F-14 ISS and ARPTT ISS) These 
systems were reviewed and examined under each of the following topics:' 

1. Task module definition 

2. Actuation cirtteria 

3. Performance measurement 

4. Scoring schemts 

5. Instructional support actions 

6. Instructor/Tss interaction 

This discussion does not compare th" instructor support feature Jmple- 
mentation of these systems; r^.the'r. it describes how training obiectives 
were utilized as controllers and generav.ors of information. 

Zajsk Module Definition 

Although the concept of a task module is used formally with only some 
Of these systems under investigation, it can be generalized to all of the 
systQu..* ty grouping together system functions which collectively meet a 
training objective. This has been done for each of the selected systems and 
the resulting sets of task modules are provided in the task commonality listing. 

systems differ in the manner of implementation: the AFTS® 
task modules" are embedded in Fortran code; F-14 ISS and ARPTT ISS are 
data-driven by formal task modules; and, the C-5A PMS has some functions 
defined in a fill -in- a- form manner but many other functions are expressed as 
an elegant general-purpose authoring language. In each case, however there 
is some means to control the decisions and actions of the ISS, 

The general form of the f-14 ISS and ARPTT ISS task modules is that of 
(a) a header that includes identifying information, start/stop loeic. and 
scoring factors applying to the entire task module, followed by (b) logic 
algorithms and diagnostic feedback appropriate to each step or event in th4 
task module. These shall be further amplified in a subsequent paragraph. The 
C-5A PMS treats specification of navigational profiles (i.e.. instrument 
departure, enroute. holding pattern, initial approach, and ILS) with a form 
that resembles the Cask module form, hut treats the remainder of specification 
With a block- strucrared language with many control and action options; the 
overall result resembles multiple nested subroutines which give the author 
detailed control over the ISS. & * 
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fltg^^t cask iftodul^^ for the ^"1^ ISS use the following format for 

1. ''^ype- i^°'^"iai, ern^'^Sency, tactical (approach, departure,..) 

%. ^^uie- operational "arue for the specific module (HI TACAN ...) 

3' '^^scriP^^°'^- a concis^ summary 

4. ^t^rt Co^'iitions: '^^e conditions which start the module 

5' Step Con^^^^ions: condi<^^°"s vhich stop the module 

6< ^^tfoJ^"**"*^^ MeaS'^^^"^ent/ScOi^ing: a listing of the steps in f^--. 
t^sk m^''^"-^^ and weighting/scoring factors. 

gach t^g^j piodule further brol^®" dovn into steps (measurable events) 
of the f°^-'-°^itig io-^^^\ 

\- ^0.: a unil*^^ sequent® number 

2- ^escri?^^^'^: statement of Performance requirements 
3. Start '^^""^itions- the cor^ditions which start measurement of the 
Step 

Stol Cf^dtti^oj^g: ' conditions which stop the step 
$. ^iagno^^^*^^ : i"^' "^^e feedback of performance, or other instruc- 
tional ^°^ions, • "^"^ on measurement within the step. 

Conseq^^^^^y, the tot^^^ task module ig composed of the header and any 
number ^^^Ps oi- ^^^'^ts, et^^ this Hia^ner, quite general flight task modules 
can he^P^'^^fied. 

^^Hipfiiri^°"* an A^^^r Takeoff Climb Task Module for the C-5A PMS 
consist^ °^ the following scatements: 

A^^^ Take cumb 

MONiTOt^ "AJ-tER TAKEOPF/CLII^" CHECKLIST 

Monttoi^ "^Q-nor^°n enrout^" enroute profile 

^EN AI-TITuSe 7 lOOOO 

Oil MAlT^j^L-C^^^N-pRESS < -10 

0^ MA^^^J^L-c^^^N-pREss > lo then 
Aj-xEE 1^^ Seconds 

ElJTEE t^^L^CTlOU 950 



Ha.lbuiJ^'^-^^n text 

tlA.LFlJtlCJ'^^ON TEXT 

HalfuN^^i^ion text 

HALF(Jt^^'^'^°N TEXT 
WHEN {^950 ^ "ACTIVE" ^^^^ 
AFTEE SECONDS 

cleaii ^^^Ll.nJNCTIOf ^so 



"WHEN ALTITUDE EXCEEDS 10000 FT 
"OR MANUAL CABIN PRESS IS NOT NORM" 
"THEN AFTER 2 MINUTES" 

ENTER MALFUNCTION 950" 



HALFtJ^^^^•^°N TEXT 

MalfuN^^i^^on text 
HalfuiJ^^^on text 



"WHEN MALFUNCTION 950 IS SET IN" 
"THEN AFTER 1 MINUTE " 

CLEAR MALFUNCTION 950" 



ERIC 



th®^ ^tXirout:^ P^^f lie i^^^^^enced above would be defined through the 

^se of ^ ^^^^ sh<5^ Pig^^^® ^^1; th^ i^eferenced checklist will be described 
later. 
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Figure F-1. Enroute Route Structure Definition Form Sheets (Sheec A of 5). 
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Figure F-1 . Enroute Route Structure Definition Form Sheets (Sheet 5 of 5). 
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procedural Tasks 



The T-IU ISS procedural task modules are of the following form for 
header information: 

1. Type: Procedures task module (no.>:il, emergency, weapons) 

2. Name: Specific name (e.g, takeoff checklist) 

3. Start conditions: conditions for str.rting the task module 

4. Stop conditions: conditions for stopping the task module 

5. Scoring: lleasuremont and scoring is done at the task module level 
rather that at the step level. Measurement consists of critical 
event measures (e.g., errors of omission or commission), mandatory 
measures (specific important switches or events) , optional measures, 
and sequence measuies. 

The remcini">ig part;3 of the task module describe the steps as follows: 

1. Step no,: unique sequence number 

2. Description: statement of the procedural activity 

3. Contingencies: the events which must have taken place prior to the 
initiation of this step. 

4. Events: a list of events appropriate to this step, including steps 
which are "correct" and those which are "incorrect" 

5. Diagnostics: feedback to the instructor indicating incorrect 
actions 

In comparison, the procedural modules definition form for the C-SA PMS 
is shown in Figure F-2, and a sample procedural specification for the Cruise 
Checklist is shown in Figure F-3. The two methods of task module specifi- 
cation are quite similar, although the C-5 PMS method provides a method for 
crew/individual measurement, and a different scheme for specifying sequences of 
actions. In that scheme, steps are grouped into blocks identified by BEGIN 
and END labels. Blocks may be nested within other blocks, and each block is 
delimited BEGIN SEQ ... END or BEGIN NSQ ... END to indicate whether the 
included steps are to be performed sequentially or in arbitrary order. When 
this scheme is used, one does not have to explicitly identify specific actions 
which must precede a specific event. 

Concurrent Modules. 

In practice, task modules may occur in sequence, one after the other, 
as TAKEOFF follows TAXI, etc. However, task modules, or steps within them, 
may occur concurrently, in parallel or overlapping fashion. In partic-^^oc it 
may be desirable to define an "umbrella'* task module that is active during an 
entire training mission; this module would continuously t- it for abnormal 
conditions which might occur at any time (e.g, crash, over-g of aircraft, 
navigation outside the defined gaming area). 
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Figure Procedural Modules Definition Form (Sheet 1 of 5), 
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Figure F~2 . Procedural Modules Definition Form (Sheet 2 of 5) . 
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Figure F-2 . Procedural Modules Definition Form (Sheet 3 of 5). 
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Figure F--2. Procedural Modules Definition Form (Sheet 5 of 5). 
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PILOT CHCCMLkdT **C9IUI0C 

FlUEHAhC: NC J )OV0O#r». CP 

Mum\^ rin£:So mnuTCs 

CLASS: ChECHUBT 

GUDCLA8B: NCmhAL 

PAmV: PILOT 

NElOHrir4Q: 10 

PnOC CRIT: I. 00 

mNl^tun x: lo 

P^TV: COPIIUOT 

NEIOHTMO: 1. 0 

PROC CRIT: I. 00 

niNihOn X: 70 

PAntY: CREW-COORD 

WEIOHTIMO: 1. 0 

PRQC CRIT: I. 00 

MINIMUM X: 70 

TYPg: SEO 

INSTRUCTOR: PC 

TC-I PAGE: 2-96 

DEF-STRT-I: LiNMi FROtI -AFTER T/O Ct-IHB- 

9TRT-TXT-I: -AFTER T/O CLIMD" CHECKLIST COMPLETED 

DEF-STOP-I: P-RADAR-ALT-8W»»'0FF* 

AFTER: 300 8ECONO0 

STOP-TXT-I: PILOTS RADAR ALTIMETER OFF ♦ fl HINUTEB 

6TEPNUMDER: 01 

STEP: "OEOtN N8Q- 

6TEPTEXT: OEOIM NSO LANDING LIGHTS OFF 



5TEPNUM8ER: 03 i <LEVEL 1 ) 

3TEP: L<*LANDINO-LIGHT^SU«'*OFF'* 

STEP TEXT: LEFT LANDING LIGHT OFF 

PARTY: PILOT 

POIMTS: 3 

STEP CRIT: 1.0 

PARTY: COPILOT 

POINTS: 3 

STEP CRIT: 1.0 

PARTY: CREW^COORD 

POINTS: 3 

6TEP CRIT: 100 

STEPNUMOER: 03 i (LPVEL 1 ) 

STEP : NOSE-LAND I NO-L I GMT •* OFF - 

STCPTEXT: NOSE LANDING AND TAXI LIGHTS OFF 

PARTY: PILOT 

POINTS: 3 

STEP CRIT: 1. 0 

PARTY: COPILOT 

POINTS: 3 

STEP CRIV: I. 0 



^ILEMAflg: NCllOVOOtP. CP 



5/9/63 19: 50: 47 PAGE I 



PARTY: 
POINTS: 
STEP CRIT: 

STEPNUriDER: 

STEP: 

BTEPTEXT: 

PARTY: 

POINTS: 

STEP CRIT: 

PARTY: 

POINTS: 

STEP CRIT: 

PARTY: 

POINTS: 

STEP CRIT: 

BTEPNUMOER: 

STEP: 

8TEPTEXT: 

BTEPNtirtDER: 

STEP: 

BTEPTEXT: 

STEPNUMDER: 

STEP: 

STEPTEXT: 

STEPNUMDER: 

STEP: 

STEPTEXT: 

WHEN-IF: 

hOMITOR-1: 

hONlT-TXTl: 

STEPNUMDER: 

STEP: 

STEPTEXT: 

PARTY: 

POINTS: 

STEP CRIT: 

PARTY: 

POINTS: 

STEP CRIT; 

PARTY: 

POINTS: 

STEP CRIT: 

WHEN-IF: 
MONITOR- I: 



CrEN-COORD 
3 

1. 00 

04 I (LEVEL 1 ) 

R < AND 1 NO -LI GHT'^SW* "OTP" 
RIGHT LANDING LIGHT OFF 
PILOT 
3 

1. 0 

COPILOT 
3 

1. 0 

CREU-COORD 
3 

1. 00 

05 I < LEVEL 1 ) 
"END- 
END LANDING LIGHTS OFF 

06 

"LEADING EDOE. AND FUSELAGE LIGHTS OFF* 
LEADING EDGE. AND FUSELAGE LIGHTS OFF 

07 

•*SEAT BELT LIGHTS: AS REQUIRED- 
SEAT BELT LIGHTS: AS REQUIRED 

08 

-ALTIMETERS: STATE SETTINPS" 
ALTIMETERS: OTATK SETTINGS 

IF 

ALTlTUDE-AOLD^fiOO 

IF CRUISING ABOVE 9000 FT AOL 

09 

P-RADAR-ALT-"5L»=«-0FF" 
PILOTS RADAR ALTITUDE OFF 
PILOT 
3 

1. 0 

COPILOT 
3 

1. 0 

CREW-COORD 
3 

1. 00 
IF 

MP01>5 



f ?r?* Sample P v :>dure Specification* 



8'? 



I 



Other Mo<; iiilAo 



2. 
3, 



task mo^ulf concept a'i'dep"^^^^^^^^^ T.'"'^^ --P^- °f the 

define modules for trrininr HoiCL tt'' be sufficient to 

oth..r categories of task"';^^dules to s 'it ^t ^"Pl^'"^"*^^^ ""^y wish to create 
For example, in the C 5a Wf? ill "l^l,^^^ purposes of a specific design. 
checlcu/.s; p^ocXef Tvil^tio^r;'^^^^^ 

envelopes. Additionally a fotm«r?r ^^*f 1^1' aircraft parameter 

shown in Figure F-V specified for malfunction insertion as 

should bTU'e ft' the'dfscTeUon o^.W H° .'^ ^-plementation-specif ic and 
specification, it may L wise to m^^nf m . t"^^!!"''' of 
may be desirable, therefore to u/e « 5 independence from design details. It 
initia specific;tior T^'follotin/f^^ task module format at the time of 

attempt to derive a co^on Sil lor comlT '"^'^ ^"^^^^^^ 

systems: ^"'^ comparison between the four selected 

Description: te.= dasc^lbing A. task or st.p to be p«fo™d 

Start logic: a concise description of the i^cr^,. ,,u< u 
to identify the starting point'for the task of ^tLp' '^"^ 

!d:StJ?y^^tho^roS^irg rornr^^^°" °^ ^^^^^ ^« -ed to 

Measurement: specific measures to bo computed 
Scoring: a method for scoring/gr^iding performance 
Action; actions to be taken during the task/«to„ a, 
massages to be sent to the Instruc^o,,' ^ll4«fo4 to 'be'S^ed 
8. Comment: any comments on the above 

With Ub:Ln;e":f°'te^t' wh^^^^^^^ f T '^^"^^ 

later in design, a number of specific l^oms m^^^ 

same form can even be used f or procedurar taT n, ! T'^''^"^- ^^^^ 
SEQ - BEGIN NSQ convention of ne.ced blocks^ ""'"^ the BEGIN 

Actuation Ci;jfc.>^i. 
Since the basic function of the kernel T<3«? 4o 

the ability to respond based on crit/rT. ^l^^^f .1 '^^J'^^^'^ take action, 

ion is of central importance a.t,^.? ^ , task module definit- 

appears to be similar «cx:uacion criteria for all four systems 



5, 
6. 
7. 
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MALFUNCnOB SPECIFICATIOK WHEM <CHECK> TUEK 

[AFTER <nuttber> SECONDS J 

/<SIMPI.E MiaFIJKCTION> ) 
\<MALFUNCriON BLOCK> / 

[<MALFUNCTION TEXX>1 

CHECK :■: Arbltrtpy Boolean expreaaloe of up to four <SIATE>fl 

MALFUNCTION BLOCK :-: BEGIN <SIMPLE MALFUNCTION> 

[WHEN <CHECK> THEN f AFTER <number> SECONDS]] <5IMP1E MALFUNCTIONS 

END 

SIMPLE KALPUNCTIOM :-: / /ENTER) jy^puNCTION <nuinber> 

/ (CLEARS 

I SET MALFUNCTION <iaalfunction variable ixame> 
I TO <valiie> 

\ FLUCiUATE MALFUNCTION <iaalfunction variable tiaiae> 
/ BETWEEN <VBlue> AND <value> 
I EVERY <nuinber> SECONDS 

' RLLEASE MALFUNCTION <Hial function variable aaffie> 
MALFUNCTION TEXT MALFUNCTI ON TEXT; text ecc^ oeigd in. quotes 



Flfiu r e .F~A . Malfunction Specification. 
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The basic structure for actuation criteria is of the fom: 
<simulator variable> <relational operator> <particular value> 

where relational operators includo equal, not eaual fv- 

than or equal to less than i«c« t-yZ^ equal, greater than, greater 



module step. 



task BoduUs^'pa^sVe"':! f ^l^tf ^ , PartlcuUrly useful for navigational 

detail rtlf*?^"'' ?f * """"^^ 'P^""^ *=t>"tlon crlMrla In aufflclent 
ion occurs 'and °no"~ 1 TluT^""" ' lnst«celonal%lti:?' 
and c„„ectir Trlnf rtral^Va^d-Te'va f\?^/to -or?,?;™*" ^.--'^f"" 

^uf^ = ^ 1'* °* actuation criteria. Otharwlao. uufortSata actions 

wti» rnrrtfo: ^i^^s::^ °' ° -i^pi^' -"i:?! 

One of the instructor support features that can be trisRered baaed or, 

r:q:enn:rs ^\^%^^!-:°-^^~ i 

when they are busy or unable to observe. It may^be the StLt^bJ^^t^kl 
student during unsupervised practice or trial ch. .des. Performance Vasu^^! 
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ment can be used by the Training Manager to assess the instructional process. 
It can provide normative data to permit comparison of a student's performance 
with other previous students at the same stage of training. Performance 
measurement can also he used for the purposes of training research. Further, 
it can be used within the ISS/ATD for instructional control. Consequently, 
there are a number of reasons for including performance measurement in an ISS. 

Flight Tqsk Measurement 

The measurement of flight tasks, as it is included in the four represen- 
tative systems, depends on whether performance is to be measured over a period 
of time, or at a specific instant of time. If performance is to be measured 
over a period of time, the following should be considered: 

1. Average value (an ordinary average, may use an absolute value of a 
parameter to treat +/- variations the same) 

2. Roo^ mean- squared value (based on a squared value so +/- values are 
treated the san.-i, an indication of variability) _ 

3. Tolerance bands (within or outsj.^ 3 of a specified range) 

If performance is to be measured at an instant of time, then a "snapshot" 
is taken of the value of flight task parameters at the designated time. 

Performance criteria, to be presented along with student performance, 
may be based on the performance of previous students. This requires that data 
are accumulated with each student, statistical analyses are performed, and 
normative performance criteria are updated at intexrvals. 

For example, tbo C-5A PMS makes the following measurements during an 
instrtunent departure: 

1. Correct NAVAID selected? (correctness of frequency -election) 

2. Correct HSI course selected? (correctness of course on HSI with 
the correct NAVAID selected) 

3. Centered GDI? (RMS course deviaticr iu dots) 

4. Specified ground track? (RMS grovuriU track error in KM) 

5. Specified DME arc track? (Average arc deviation error in NM) 

6. Within altitude restriction? (Altitude error at checkpoint) 

Procedural Ta s ^ Measurement 

Procedural measurement for the representative systems incorporates the 
following: 

1. Errors of omission events not performed 

2. Errors of commission unwanted events were performed 

3. Constraint errors at specified events, flight parameters were 
out of tolerance (e.g., airspeed at flaps down) 

Sequence errors events out of sequence 
«^cent of mandatory actions completed 
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6. Peroenc of opcional actions oompleCed 

7. Time to first event - e.g, beginning of checklist 

module lfvti*°aUXf Ifttu'tTr" ^-^ely at the task 

srM- ia~ H5~"- ^»r-h%<ri^n%£%:3: - 

should ^curiraTrocitrer " *ich events 

r;-L^^" — - - o'k'-r4e;r^dt i— 

procedure ^^''^^ """'^ and after each iten, in a 

i.e.. cln.lt.^l\T.^^^^^^^^ a procedure, 

flaps are lowered). °^ airspeed when 

S coring 

types of "tasks'""'^: "z^ber^r''^"™'"." "^^ S-nerated for various 

sSulator s?ssl;„s\"^e:^s1or "ZSL'wTJ task .odules, and multiple 
desired. No standard "eSod t^r th?, f/ 1 ""I »>ay be 

used__for some of the reprcsentftL'Ss r pi^se'^te^^ln'-ThrfVirCpt^ 

AFTS® 

band of performance. The points awarded were then differentiallv LiT.^ 
accordance with assigned weights to produce a proficiencrscore L "5 
an exercise. Based on the derived score the at^'cjI ^ ?^ u °^ 

-S-^-AvtS SvSd- ^^^^^ 
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F>1A ISS 



For each flight task or procedural task measurement, a score was assigned 
(in the range 2.5 to 4.0), then each score was multiplied by a weighting 
factor, and all weighted scores were summed to produce a score for each 
task module. A matrix was produced for each task module, for viewing by the 
instructor, the rows of which corresponded to each performance measure in the 
task module, and the columns included the following information: 

1. Nominal Value 

2. 4.0 Range (upper and lower measurement limits for this 

3. 3.5 Range (upper and lower measurement limits for this 

4. 3.0 Range (upper and lower measurement limits for this 

5. 2.5 Range (upper and lower measurement limits for this 

6. Measured Value 

7. Number of Measures 

8 . Gtrade 

9. Weight Factor 

Using this output , the instructor could see the desired value of a 
specific performance measure (nominal value), the range of performance that 
would yield a specific grade, the measured or actual value of performance, the 
grade which this algorithm produced, and the weighting factor for combining 
each measure grade into a grade for the entire task module. An advantage 
of this approach is that the grading algorithm is clearly displayed, and 
tY'j instructors are permitted to change Range definitions or Weight Factors 
to agree with their subjective standards. 

C-5A PMS 

The task module definitions for the C-5A PMS include the assignment 
of points to each step in a procedure, each parameter envelope, and each 
navigational profile; further, provision is made to assign the resulting 
performance measur*is to one or more crewmembers . For example, a procedure may 
yield all "possible points" if performed correctly, half of the "possible 
points" if there is a sequence violation, and no points if an omission or 
constraint violation occurs. Further, the earned points are multiplied by a 
critical! ty factor for each step to reflect differences in the seriousness of 
errors . 

Based on the points produced by the previous method, five levels of 
proficiency assessment are derived: 

1. Performance Monitorable Task Assessment Combination of individual 
scores into a single score for the performance monitorable task. There will 
be a separate score for each crew member/crew coordination associated with the 
task. 

2. Performance Monitorable Task Group Assessment Combination oi' 
scores for all tasks belonging to the same group (procedure, navigHtior.-.i 
profile, parameter). There will be a separate score for each session. 



score) 
score) 
score) 
score) 
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3. Crew Member/Crew Coordination AssessmGnt Pom-k^^.-^ 
~ry performance sc„„s fro» Uvel a £o/r«r™\.„ "r'.^rcj:: c';;or.'l! 



4. Session Assessment Combination of the pilot, cooilot fH.T,*. 
12^11^ rs:r:io„""*"""'°" sco're°?„t"a 



5. Mission Assessmenc -- Combination of hV.*. «,-^<r*-4- 
scores for both sessions . ^omomation of the proficiency assessment 

session, and then they reverse roles on the second session. 
AR?TT T?^?l 

Minimum proficiency levels were defined for each of the ARPTT tr^fnfr,. 
ohjec.iver and stored as part of the task module definitions Te TSstS 

The instructor%h«n^.H instructor during an evaluation of objectives. 

^or..t:To:Tf\^^:: ^^^^^^^^ if ^esi^ed. The 

auo used as a reference of the student^s^pr ogress Ove^a 1 i"'^'"? 'Tf ^ 
were also maintained for review bv curriculum ™' ^''"^^K''^^^^ statistics 
requirements of the syllabus ^ curriculum managers of standardization 

Ills true tlonal Sup port Act^nni^ 

dltions, communications, and recording of data Parh »f ^"^^^^^-^ ^o"" 

ISS, has thl. capability *ich dlE£e„^^acc„r<ia„c^„.t^ tS%p:?"lc"a1„' f 
cation and not In any substantial wav Som. of r , specific appll- 

su»arl.ed below without attempt to c'^I^ast W^ate^ " '"^"^ 
Malfunction Cp ntrni 

A number of malfunctions are ordinarily possible In a modem FH„T,f- 

:=ons"tbft' - "^nrLt^rry ? 

C^s^^Pc^tn==;ar-a^^ 

ofZi^Hw^ 

bTl vH^^^^^^^^^^^^ 

iLtructlo^l dut" ' ''"'"^ instructor tor other 
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Based on actuation criteria stored as part of the task module def inltion» 
the ISS can trigger the generation of displays without direct action by any 
personnel. This also offers the potential for unburdening instructional 
personnel of operator duties and freedom for instruction. Among the types of 
displays chat can be produced are: 

1. Hlsslon sequence displays (sequences of tasks) 

2. Mission plot displays (ground- referenced ^ravhics) 

3. Route charts (displays for departure, enroute, approach plates) 

4. Checklist/procedure displays (displays of predefined sequences 
together with time-stamped crew actions) 

5. Error alert displays (messages alerting instructor to errors) 

6. Proficiency assessment displays (scores/grades for specific tasks 
and total mission) 

7 . Debriefing report (data to support the debriefing period) 

I he ISS may be viewed as an information generator, of which some 
information is generated to support training- in-progress , and other information 
is generated to support external training processes. Among the external needs 
tUac may be supported are debriefing, development of performance norms, 
training management data analysis, and training research data analysis. Axi 
ex^aple of statistics accumulated by the ARPTT is shown in Table F-1- Data 
for these needs will depend on the specific overall training system, but will 
certainly include all recording of all events in a manner appropriate for 
debriefing, and will include all basic data used in developing grades for 
proficiency assessment. Specific research requirements may dictate that even 
greater perfomnance measurement detail be recorded. 



Table F^l . Precontact Statistics 
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Instruct or ISS Tn«;eract^ orf 

^ ^^l foregoing discussion is based on preprogrammed automatic 

operation of the ISS. Although this mode has a number of advantages ?t could 
iiteractTon""'/'^'" environment for instruction unless options for instructor 
interaction and override are provided. Furthermore, not all crew actions 

out-of-cockplt visual objects) can be automatically 
llTJf 1.% ^'^"ructor must provide the needed information 

It^. ? f""*"" systems provides some means for deviation from fully pre^ 
programmed and automatic operation. iui^y pre 

AFTS® 

The automated-adaptive mode is the normal mode for AFTS®, however it 
does allow the instructor to control the sequence of training Adaptive 

rmenf?f'r;it?ar'''i:t'^ * training' session with a se'lectt^'fri: 

a menu of initial conditions. For a given set of initial conditions, AFTS® can 
be paused with a FREEZE command, and either CONTINUE or RESET to the s^art fir 

ARPTT AND PI A T«?f! 

Both the ARPTT and F14 ISS provide options for selecting a fullv 
c^nstr^ucTd mode (CANNED mode), part.task trailing (PIT) mode? orfnatr^cLj 
constructed sessions produced from a menu of task modules (ISSM or ISEL 
modes). Preprogrammed insertion of malfunctions can be modified throup^h 
Tan li ^d7« Y ^""1 "^LHINCTION switches . Control ^^task Tdule seqSn'ng 
can be modified by selection of reposition options to SLEW TO or RADAR VECTOR 
TO. allowing training to begin from a new position. 

S.-5A m 

in the L'Ifn P"^^*^" J"^^ instructor with the capability to intervene 
tf cwJfr? sequence of events and allows modification of the selection 

of checklists, procedures and malfunctions, and alteration of the selection of 
displays- of mission information. During training, the instructor may exercisf 

of^if^ctW "'^'"l^ provided to control the insertion and removal 

The preceding discussion is only exemplary and does not eive a full 
presentation of the options for control provided for tach System It Is 
i^ir^^'/rT*'^'^ "^^^ suggestive of types of control which the instructor maj 
ill Tcc^ ^"Sgestive of the interaction that must be specified for tach new 

ISS. The ISS must unburden the instructor, and may do this through the use of 
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Cor^cluslons 

A view has been taken that the ISS is a progranuflable controller and a 
generator of information, and although the four representative ISSs vary in 
the way they are implemented, all fit the same general model. The method of 
specifying the program for the ISSs varies, but the task module concept can be 
used for initial specification for any ISS, This implies a data-driven 
system, but specification in this manner still peirmits a large degree of 
design freedom. The types of actuation criteria included in an ISS determines 
how "smart" the ISS can be in behaving "intelligently" in controlling Instruc- 
tional events and features. The ISS can be the generator of a large amount of 
different types of information, and this capability depends on the manner in 
which performance measurement is implemented and the types of data recorded 
and displayed. Although all four of the systems provide a preprogrammed 
automatic mode, each provides some manner for instructor control and override, 
providing a degree of flexibility in use of the ISS for tailored instruction. 
All provide Instructional support through the control of performance measure- 
ment, scoring, displays, malftmctions, communications and data recording. These 
four systems provide a range of examples that characterize the current tech- 
nology and provide a basis for the generation of future specifications. 
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GLOSSARY OF TERMS 



AIRCREW TRAINING DEVICE (AID): A term that refers to synthetic training devices 
(simulators) used in support of aircrew training programs. These devices 
range from simple procedures trainers to more complex training systems. 

ALGORITHM: A precise characterization of a method for solving a problem or 
achieving a goal, e.g., a sequence of actions terminating in a solution* 

BRIEF: Review of events, objectives and procedures with aircrew and instruc- 
tional staff prior to simulator session. 

CHECKLIST: A series of distinct actions to be performed at discrete times. 

CHECKRIDE: A mission or profile in which the computer monitors the student 
performance, usually from takeoff to final landing, without intervention by the 
instructor. 

CONTINUATION TRAINING: Training conducted routinely in operational squadrons, 
or proficiency training conducted periodically. 

CONVERSION TRAINING: Initial qualifying training for a particular type 
of weapon system. 

DATA-DRIVEN: A system that relies on general software which acts upon a 
database, such t^at a change to the database would not affect a change to 
the software. 

DEBRIEF: Review of event results with aircrew and instructors subsequent 
to simulator session. 

INITIAL CONDITIONS (I.C.s): A set of conditions or starting points for each 
training scenario. These include variables such as airspeed, altitude, fuel 
load, etc. 

INITIALIZATION: Initialization involves specifying, usually from the instruc- 
tor/operator console, the parameters of interest and their values for posi- 
tioning and configuring an ATD within a gaming area. 

INSTRUCTOR SUPPORT FEATURE (ISF) : Features provided by the Instructional 
Support System (IS) to aid the ATD instructor in conducting the training 
exercise. 

INSTRUCTIONAL SYSTEMS DEVELOPMENT (ISD) : Procedural approach to the analysis 
of training requirements and the development of training programs and systems. 

INSTRUCTOR/OPERATOR STATION (lOS) : The aircrew training device man-machine 
interface where active control and monitoring of training events occurs. 
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INSTRUCTOR SUPPORT SYSTEM (ISS) : Automated system within the ATD designed to 
aid the Instructor in performing the training function. 

^^"^^J'^Jf^^".^^ ^^^^^ STATEMENT (MENS): A statement prepared by HQ USAF 
S^f nnrl^.ir ^^''^r" ""^^ "^^^ ^"'^ ^ improved mission capability, 

of ^>fo A ^ t ^ ^^^^a °" °^ ^°Ns and is prepared if the Secretary 

approach ^''^^ ""^ Secretary of Defense must approve the need and the solution 

MODULARITY: Property of a system which allows individual units to be added, 
deleted, or modified, without affecting remainder of that system. 

OFF-BOARD STATION: Instructor/operator station which is outside cockpit. 
OFF-LINE: Any action not associated with active training on the simulator. 
ON-BOARD STATION: Instructor/operator station which is inside cockpit, 
active^ training""^^^*^ directly by a computer, usually in association with 

OPERATIONAL FLIGHT TRAINER (OFT): A device that dynamically simulates the 
"^''J'"^^^^''^ °^ designated aircraft to train flight crews in 

cockpit procedures, instrument flight procedures, emergency procedural., commu- 
nications and navigation procedures, and includes limited mission execution. 

^tT ^fJ^; operation air refueling, radar operations, etc.) to be practiced 
and a high degree of skill developed independently of other mission tasks. 

PERFORMANCE MEASUREMENT SYSTEM (PMS) : The computer-based monitoring, recording 
processing and displaying of objective, quantitative information for describing 
and diagnosing student performance. i-^^uj-ug 

PROGRAM MANAGEMENT DIRECTIVE (PMD) : The official HQ USAF management directive 

and to LMcf^f ^° implementing and participating commands 

and to satisfy documentation requirements. «- e» 

PROGRAMMED MISSION SCENARIOS: Highly structured sets of events that are 
caused to occur automatically, under com,?uter control. 

SAMPLING RATE: The temporal frequency at which a stated variable (parameter) 
may be recorded or examined by an automated performance measurement system, 

SCENAR^IO: A predefined sequence of training events used to exercise the 
capabxlxties of an ATD in a specific area of intended training usage. 

SPECIFICATION: Statement describing the device to be built in terms of its 
functions and characteristics. 

STATEMENT OF OPERATIONAL NEED (SON): A general statement of requirements 
prepared by one of the Air Force Major Commands. 
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syllabus: Course of study 

TASK MODULE: User- oriented building blocks that correspond to the operational 
training requirements which have a direct correlation to a group of files 
which make up the data base for a modular data base driven jystem. 

TRAINING OBJECTIVES: Explicit statements of the goals of training including 
tasks to be performed, the performance standards for each task, and the 
conditions under which those tasks are to be performed. 

TRAINING REQUIREMENTS: General statements of task performance skills required 
for operational proficiency. 

UNDERGRADUATE PILOT TRAINING; Initial pilot flight training. 

WEAPON! SYSTEM TRAINER (WST): A device which provides a synthetic flight 
and tactics environment in which aircrews learn, develop, and improve the 
techniques associated with their crew position in a specific aircraft, and 
operate individually or as a team in the execution of simulated missions* 
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ABBREVIATIONS AND ACRONYMS 



AAI air-to-air Intercept 

AFB Air Force Base 

AFHRL Air Force Human Resources Laboratory 

AFLC Air Force Logistics Command 

AFR Air Force Regulation 

AFSC Air Force Systems Command 

AFTS® Automated Flight Training System 
ARPTT ' aerial re£uellng part- task trainer 

ASD Aeronautical Systems Division 

ATC Air Training Command 

ATD aircrew training device 

BVR beyond visual range 

CCA carrier controlled approach 

CDRL contract data requirements list 

CPT cockpit procedures trainer 

CRT cathode ray ttibe 

DBHS data base management system 

DO Director o£ Operations 

OR Director o£ Requirements 

DRF pnal Role Fighter 

EHET Engineering Equipment and Training 

GAR ground attack radar 

GAT ground attack tactical- 

GCA ground controlled approach 

HQ USAF Headquarters, United States Air Force 

Hz hertz 

I.e. Initial condition 

ILS Instrument Landing System 

I/O Input/output 

lOS Instructor/operator station 

lOT&E Initial operational test and evaluation 

IF Instructor pilot 

ISD Instructional system development 

ISF Instructor support feature 

ISS Instructor Support System 

KB kilobyte 
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MAC Military Airlift Command 

MAJCOM major coininand 

MB megabyte 

MCU malfunction control unit 

MENS Mission Element Need Statement 

MIL-STD Military Standard 



OFT operational flight trainer 

OT&E operational test & evaluation 

PIDS prime item development specification 

PM program manager 

PMD Program Management Directive 

PMP Program management Plan 

PMS performance measurement system 

PTT part- Cask trainer 

R&D research and development 

SAC Strategic Air Command 

SECU simulation exercise control unit 

SID standard instniment departure 

SIMCERT Simulator Certification Program 

SimSPO Simulator Systems Program Office 

SLOC source lines of code 

SON Statement of Need 

SOW Statement of Work 



TAC Tactical Air Command 

TACAN tactical air navigation 

TM task module 

UPT Undergraduate Pilot Training 

WST weapon system trainer 
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